agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH v19 6/8] Row pattern recognition patch (docs). 378+ messages / 2 participants [nested] [flat]
* [PATCH v19 6/8] Row pattern recognition patch (docs). @ 2024-05-14 23:26 Tatsuo Ishii <ishii@postgresql.org> 0 siblings, 0 replies; 378+ messages in thread From: Tatsuo Ishii @ 2024-05-14 23:26 UTC (permalink / raw) --- doc/src/sgml/advanced.sgml | 82 ++++++++++++++++++++++++++++++++++++ doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++ doc/src/sgml/ref/select.sgml | 38 ++++++++++++++++- 3 files changed, 172 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 755c9f1485..b0b1d1c51e 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -537,6 +537,88 @@ WHERE pos < 3; <literal>rank</literal> less than 3. </para> + <para> + Row pattern common syntax can be used to perform row pattern recognition + in a query. The row pattern common syntax includes two sub + clauses: <literal>DEFINE</literal> + and <literal>PATTERN</literal>. <literal>DEFINE</literal> defines + definition variables along with an expression. The expression must be a + logical expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. The expression may comprise column references + and functions. Window functions, aggregate functions and subqueries are + not allowed. An example of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns the price column in the + previous row if it's called in a context of row pattern recognition. Thus in + the second line the definition variable "UP" is <literal>TRUE</literal> + when the price column in the current row is greater than the price column + in the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when + the price column in the current row is lower than the price column in the + previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + certain conditions. For example following <literal>PATTERN</literal> + defines that a row starts with the condition "LOWPRICE", then one or more + rows satisfy "UP" and finally one or more rows satisfy "DOWN". Note that + "+" means one or more matches. Also you can use "*", which means zero or + more matches. If a sequence of rows which satisfies the PATTERN is found, + in the starting row of the sequence of rows all window functions and + aggregates are shown in the target list. Note that aggregations only look + into the matched rows, rather than whole frame. On the second or + subsequent rows all window functions are NULL. Aggregates are NULL or 0 + (count case) depending on its aggregation definition. For rows that do not + match on the PATTERN, all window functions and aggregates are shown AS + NULL too, except count showing 0. This is because the rows do not match, + thus they are in an empty frame. Example of a <literal>SELECT</literal> + using the <literal>DEFINE</literal> and <literal>PATTERN</literal> clause + is as follows. + +<programlisting> +SELECT company, tdate, price, + first_value(price) OVER w, + max(price) OVER w, + count(price) OVER w +FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> +<screen> + company | tdate | price | first_value | max | count +----------+------------+-------+-------------+-----+------- + company1 | 2023-07-01 | 100 | 100 | 200 | 4 + company1 | 2023-07-02 | 200 | | | 0 + company1 | 2023-07-03 | 150 | | | 0 + company1 | 2023-07-04 | 140 | | | 0 + company1 | 2023-07-05 | 150 | | | 0 + company1 | 2023-07-06 | 90 | 90 | 130 | 4 + company1 | 2023-07-07 | 110 | | | 0 + company1 | 2023-07-08 | 130 | | | 0 + company1 | 2023-07-09 | 120 | | | 0 + company1 | 2023-07-10 | 130 | | | 0 +(10 rows) +</screen> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 17c44bc338..8dbab31300 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -23124,6 +23124,7 @@ SELECT count(*) FROM sometable; returns <literal>NULL</literal> if there is no such row. </para></entry> </row> + </tbody> </tgroup> </table> @@ -23163,6 +23164,59 @@ SELECT count(*) FROM sometable; Other frame specifications can be used to obtain other effects. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>RESPECT NULLS</literal> or diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 066aed44e6..8f18718d58 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -969,8 +969,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1077,6 +1077,40 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls + how to proceed to next row position after a match + found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the + default) next row position is next to the last row of previous match. On + the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next + row position is always next to the last row of previous + match. <literal>DEFINE</literal> defines definition variables along with a + boolean expression. <literal>PATTERN</literal> defines a sequence of rows + that satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. If the variable is not defined in + the <literal>DEFINE</literal> clause, it is implicitly assumed + following is defined in the <literal>DEFINE</literal> clause. + +<synopsis> +<literal>variable_name</literal> AS TRUE +</synopsis> + + Note that the maximu number of variables defined + in <literal>DEFINE</literal> clause is 26. + +<synopsis> +[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ] +PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...] +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the behavior of <firstterm>window functions</firstterm> appearing in the query's -- 2.25.1 ----Next_Part(Wed_May_15_09_02_03_2024_008)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v19-0007-Row-pattern-recognition-patch-tests.patch" ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
* [PATCH] Reproduce filtering issue. @ 2026-03-23 11:50 Antonin Houska <ah@cybertec.at> 0 siblings, 0 replies; 378+ messages in thread From: Antonin Houska @ 2026-03-23 11:50 UTC (permalink / raw) --- contrib/test_decoding/expected/filtering.out | 74 ++++++++++++++ contrib/test_decoding/specs/filtering.spec | 101 +++++++++++++++++++ src/backend/executor/nodeModifyTable.c | 2 + src/backend/replication/logical/snapbuild.c | 3 + src/test/isolation/isolationtester.c | 9 +- 5 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 contrib/test_decoding/expected/filtering.out create mode 100644 contrib/test_decoding/specs/filtering.spec diff --git a/contrib/test_decoding/expected/filtering.out b/contrib/test_decoding/expected/filtering.out new file mode 100644 index 00000000000..6ba9690509f --- /dev/null +++ b/contrib/test_decoding/expected/filtering.out @@ -0,0 +1,74 @@ +Parsed test spec with 5 sessions + +starting permutation: s1_assign_xid s3_repack s2_assign_xid s1_rollback s4_insert s5_wakeup_snapbuild s2_rollback s5_wakeup_insert_speculative s4_insert_commit s5_wakeup_repack +injection_points_attach +----------------------- + +(1 row) + +injection_points_attach +----------------------- + +(1 row) + +step s1_assign_xid: + BEGIN; + CREATE TABLE c(i int); + +step s3_repack: + REPACK (CONCURRENTLY) a; + <waiting ...> +step s2_assign_xid: + BEGIN; + CREATE TABLE d(i int); + +step s1_rollback: + ROLLBACK; + +step s4_insert: + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; + <waiting ...> +step s5_wakeup_snapbuild: + SELECT injection_points_wakeup('snapbuild-full'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s2_rollback: + ROLLBACK; + +step s5_wakeup_insert_speculative: + SELECT injection_points_wakeup('insert-speculative-before-confirm'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s4_insert: <... completed> +step s4_insert_commit: + COMMIT; + +step s5_wakeup_repack: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step s3_repack: <... completed> +injection_points_detach +----------------------- + +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/contrib/test_decoding/specs/filtering.spec b/contrib/test_decoding/specs/filtering.spec new file mode 100644 index 00000000000..a02c9f8facb --- /dev/null +++ b/contrib/test_decoding/specs/filtering.spec @@ -0,0 +1,101 @@ +setup +{ + CREATE TABLE a(i int primary key, j int) WITH (autovacuum_enabled = off); + INSERT INTO a(i, j) VALUES (1, 1), (2, 2); + CREATE TABLE t(i int primary key); + INSERT INTO t(i) VALUES (1); + CREATE EXTENSION injection_points; +} + +session s1 +step s1_assign_xid +{ + BEGIN; + CREATE TABLE c(i int); +} +step s1_rollback +{ + ROLLBACK; +} + +session s2 +step s2_assign_xid +{ + BEGIN; + CREATE TABLE d(i int); +} +step s2_rollback +{ + ROLLBACK; +} + +session s3 +setup +{ + SELECT injection_points_attach('snapbuild-full', 'wait'); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +step s3_repack +{ + REPACK (CONCURRENTLY) a; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); + SELECT injection_points_detach('snapbuild-full'); +} + +session s4 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('insert-speculative-before-confirm', 'wait'); +} +step s4_insert +{ + BEGIN; + INSERT INTO t(i) + SELECT max(i) + 1 FROM t ON CONFLICT (i) DO UPDATE SET i=EXCLUDED.i; +} +step s4_insert_commit +{ + COMMIT; +} +teardown +{ + SELECT injection_points_detach('insert-speculative-before-confirm'); +} + +session s5 +step s5_wakeup_snapbuild +{ + SELECT injection_points_wakeup('snapbuild-full'); +} +step s5_wakeup_insert_speculative +{ + SELECT injection_points_wakeup('insert-speculative-before-confirm'); +} +step s5_wakeup_repack +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +permutation +# Bring the snapshot builder to the FULL_SNAPSHOT state. +s1_assign_xid +s3_repack +s2_assign_xid +s1_rollback +# Perform the speculative insert, but no confirmation so far. The snapshot +# builder should decode it. +s4_insert +# Let the snapshout builder achieve CONSISTENT state and finish the setup. +s5_wakeup_snapbuild +s2_rollback +# While REPACK is waiting on repack-concurrently-before-lock, let the insert +# get confirmed. Relation filtering is now enabled. +s5_wakeup_insert_speculative +s4_insert_commit +# REPACK should now decode the speculative insert and decode the speculative +# insert (with the confirmation record filtered out). +s5_wakeup_repack diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 680c29f35d5..6d5482e5746 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1232,6 +1232,8 @@ ExecInsert(ModifyTableContext *context, slot, arbiterIndexes, &specConflict); + INJECTION_POINT("insert-speculative-before-confirm", NULL); + /* adjust the tuple's state accordingly */ table_tuple_complete_speculative(resultRelationDesc, slot, specToken, !specConflict); diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index fbdd4600a2b..883e5b6e18f 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -141,6 +141,7 @@ #include "storage/procarray.h" #include "storage/standby.h" #include "utils/builtins.h" +#include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/snapmgr.h" #include "utils/snapshot.h" @@ -1390,6 +1391,8 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn builder->state = SNAPBUILD_FULL_SNAPSHOT; builder->next_phase_at = running->nextXid; + INJECTION_POINT("snapbuild-full", NULL); + ereport(LOG, errmsg("logical decoding found initial consistent point at %X/%08X", LSN_FORMAT_ARGS(lsn)), diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c index 440c875b8ac..8f17ee412c9 100644 --- a/src/test/isolation/isolationtester.c +++ b/src/test/isolation/isolationtester.c @@ -216,15 +216,22 @@ main(int argc, char **argv) * exactly expect concurrent use of test tables. However, autovacuum will * occasionally take AccessExclusiveLock to truncate a table, and we must * ignore that transient wait. + * + * If the session's backend is blocked, and if its background worker is + * waiting on an injection point, we assume that the injection point is + * the reason for the backend to be blocked. That's what we check in the + * second query of the UNION. XXX Should we use a separate query for that? */ initPQExpBuffer(&wait_query); appendPQExpBufferStr(&wait_query, + "WITH blocking(res) AS (" "SELECT pg_catalog.pg_isolation_test_session_is_blocked($1, '{"); /* The spec syntax requires at least one session; assume that here. */ appendPQExpBufferStr(&wait_query, conns[1].backend_pid_str); for (i = 2; i < nconns; i++) appendPQExpBuffer(&wait_query, ",%s", conns[i].backend_pid_str); - appendPQExpBufferStr(&wait_query, "}')"); + appendPQExpBufferStr(&wait_query, "}') UNION " + "SELECT pg_catalog.pg_isolation_test_session_is_blocked(pid, '{}') FROM pg_stat_activity WHERE leader_pid=$1) SELECT bool_or(res) FROM blocking"); res = PQprepare(conns[0].conn, PREP_WAITING, wait_query.data, 0, NULL); if (PQresultStatus(res) != PGRES_COMMAND_OK) -- 2.47.3 --=-=-=-- ^ permalink raw reply [nested|flat] 378+ messages in thread
end of thread, other threads:[~2026-03-23 11:50 UTC | newest] Thread overview: 378+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-05-14 23:26 [PATCH v19 6/8] Row pattern recognition patch (docs). Tatsuo Ishii <ishii@postgresql.org> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at> 2026-03-23 11:50 [PATCH] Reproduce filtering issue. Antonin Houska <ah@cybertec.at>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox