public inbox for [email protected]
help / color / mirror / Atom feedFrom: Gleb Kashkin <[email protected]>
To: Alexander Korotkov <[email protected]>
Cc: pgsql-hackers <[email protected]>
Subject: Re: Bug in asynchronous Append
Date: Fri, 10 Jul 2026 13:56:46 +0300
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAPpHfduMOTnV5Zj2KGJ7zanL_10QvccZHtPUaDfJvBhsh9axnQ@mail.gmail.com>
References: <CAPpHfduMOTnV5Zj2KGJ7zanL_10QvccZHtPUaDfJvBhsh9axnQ@mail.gmail.com>
Alexander Korotkov писал(а) 2026-07-04 01:00:
> Hi!
>
> ExecReScanAppend() unconditionally resets callback_pending for all
> AsyncRequests. The problem is that postgres_fdw keeps its own
> knowledge for the same fact: PgFdwConnState.pendingAreq – a pointer
> to "pending async request" for a given connection. That connection
> can be shared by several partitions/foreign tables (postgres_fdw
> caches one connection per server+usermapping pair). The blind reset in
> nodeAppend.c only touches the local AsyncRequest.callback_pending; it
> never touches PgFdwConnState.pendingAreq, which correctly points to
> the still-dangling request.
>
> Later, when another partition sharing that same connection gets its
> own ReScan (for instance, its chgParam changed because of the LATERAL
> parameter, and it already has a cursor open), it sends "CLOSE cursor"
> via pgfdw_exec_query(). Before sending any new command on the
> connection, that function first drains whatever request is still
> outstanding on it:
>
> if (state && state->pendingAreq)
> process_pending_request(state->pendingAreq);
>
> And process_pending_request() starts with:
>
> Assert(areq->callback_pending);
>
> – which fails, because the flag was corrupted some rounds earlier.
>
> The attached patch contains both the reproduction case and the fix.
> The fix postpones the reset of the callback_pending flag to
> ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup
> along with ExecReScan(), which completes the async fetch.
>
> ------
> Regards,
> Alexander Korotkov
> Supabase
Hi!
It seems, that there is a major issue with the patch, the draining
doesn't
throw away the result in cases when it is required. Consider this case:
-- Expose stale rows from a pending async request that was valid for a
-- previous rescan but is pruned out for the current one.
CREATE VIEW base_tbl2_slow AS
SELECT t.a, t.b, t.c
FROM base_tbl2 t, LATERAL pg_sleep(0.2);
ALTER FOREIGN TABLE async_p2 OPTIONS (SET table_name 'base_tbl2_slow');
SELECT o.x, s.a
FROM (VALUES (2505), (3505)) o(x),
LATERAL (
SELECT a
FROM async_pt
WHERE a = o.x OR (o.x = 2505 AND a = 1505)
LIMIT 1
) s
ORDER BY o.x;
x | a
------+------
2505 | 1505
3505 | 2505
(2 rows)
1) Append executes two async fscans, one of which is slow
2) The first fscan finishes and the append execution stops due to LIMIT
1
3) Second fscan async request is still pending
4) Next append rescan prunes async_p2 for new outer value
5) Old async request for async_p2 is allowed to drain (receive a
callback)
6) The stale tuple from async_p2 is accepted by Append, no
pruning/filter check
7) We get a row that should be impossible (3505 | 2505) in this query
See full reproducer in the attached patch.
Attachments:
[text/x-diff] v1-regress.patch (4.9K, ../[email protected]/2-v1-regress.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index e90289e4ab1..eebdf1c4ad8 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11661,6 +11661,22 @@ SELECT * FROM result_tbl ORDER BY a;
(3 rows)
DELETE FROM result_tbl;
+-- Check that a pending async request on a shared connection isn't corrupted
+-- when the Append is rescanned with the subplan holding it pruned out. Here
+-- async_p2 and async_p3 share a connection, and per outer row exactly one of
+-- them is pruned while async_p1 (a different connection) always matches; the
+-- inner LIMIT leaves the other shared-connection request outstanding across
+-- the rescan, and reusing that connection for the next round must drain it
+-- rather than trip over stale state.
+SELECT o.x FROM (VALUES (2505), (3505)) o(x),
+ LATERAL (SELECT a FROM async_pt WHERE a = 1505 OR a = o.x LIMIT 1) s
+ORDER BY o.x;
+ x
+------
+ 2505
+ 3505
+(2 rows)
+
-- Test COPY TO when foreign table is partition
COPY async_pt TO stdout; --error
ERROR: cannot copy from foreign table "async_p1"
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index dfc58beb0d2..c18b54f1756 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3997,6 +3997,38 @@ INSERT INTO result_tbl SELECT * FROM async_pt WHERE b === 505;
SELECT * FROM result_tbl ORDER BY a;
DELETE FROM result_tbl;
+-- Check that a pending async request on a shared connection isn't corrupted
+-- when the Append is rescanned with the subplan holding it pruned out. Here
+-- async_p2 and async_p3 share a connection, and per outer row exactly one of
+-- them is pruned while async_p1 (a different connection) always matches; the
+-- inner LIMIT leaves the other shared-connection request outstanding across
+-- the rescan, and reusing that connection for the next round must drain it
+-- rather than trip over stale state.
+SELECT o.x FROM (VALUES (2505), (3505)) o(x),
+ LATERAL (SELECT a FROM async_pt WHERE a = 1505 OR a = o.x LIMIT 1) s
+ORDER BY o.x;
+
+-- Expose stale rows from a pending async request that was valid for a
+-- previous rescan but is pruned out for the current one.
+CREATE VIEW base_tbl2_slow AS
+ SELECT t.a, t.b, t.c
+ FROM base_tbl2 t, LATERAL pg_sleep(0.2);
+
+ALTER FOREIGN TABLE async_p2 OPTIONS (SET table_name 'base_tbl2_slow');
+
+SELECT o.x, s.a
+FROM (VALUES (2505), (3505)) o(x),
+ LATERAL (
+ SELECT a
+ FROM async_pt
+ WHERE a = o.x OR (o.x = 2505 AND a = 1505)
+ LIMIT 1
+ ) s
+ORDER BY o.x;
+
+ALTER FOREIGN TABLE async_p2 OPTIONS (SET table_name 'base_tbl2');
+DROP VIEW base_tbl2_slow;
+
-- Test COPY TO when foreign table is partition
COPY async_pt TO stdout; --error
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 987358e27fa..6a5a14cd576 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -469,7 +469,21 @@ ExecReScanAppend(AppendState *node)
{
AsyncRequest *areq = node->as_asyncrequests[i];
- areq->callback_pending = false;
+ /*
+ * Leave a request that is still marked as pending a callback
+ * alone: it may genuinely still be in flight, or it may have an
+ * unconsumed result already sitting on a connection shared with
+ * another subplan (as can happen with postgres_fdw). Blindly
+ * clearing callback_pending here would desync our bookkeeping
+ * from the async-capable node's own, which can lead it to
+ * mishandle that connection later (e.g. postgres_fdw asserts that
+ * a request it still considers in-process has callback_pending
+ * set). Such a request is instead drained lazily, right before
+ * it would be reused, in ExecAppendAsyncBegin().
+ */
+ if (areq->callback_pending)
+ continue;
+
areq->request_complete = false;
areq->result = NULL;
}
@@ -915,7 +929,25 @@ ExecAppendAsyncBegin(AppendState *node)
AsyncRequest *areq = node->as_asyncrequests[i];
Assert(areq->request_index == i);
- Assert(!areq->callback_pending);
+
+ /*
+ * This request may still be marked as pending a callback, if
+ * ExecReScanAppend() left it alone because it might have been
+ * genuinely in flight (or had an unconsumed result waiting on a
+ * connection shared with another subplan). Drain it now, before
+ * reusing it: ExecReScan() lets the async-capable node settle any
+ * such outstanding state (e.g. postgres_fdw's
+ * postgresReScanForeignScan() will wait for an in-progress request on
+ * its own connection and consume its result), after which it's safe
+ * to reset our own bookkeeping and issue a fresh request.
+ */
+ if (areq->callback_pending)
+ {
+ ExecReScan(node->appendplans[i]);
+ areq->callback_pending = false;
+ areq->request_complete = false;
+ areq->result = NULL;
+ }
/* Do the actual work. */
ExecAsyncRequest(areq);
view thread (21+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: Bug in asynchronous Append
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox