pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Matheus Alcantara <matheusssilv97@gmail.com>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Asynchronous MergeAppend
Date: Mon, 02 Nov 2026 09:58:40 +0300
Message-ID: <0bf760a18b139ece9f447c2082499e6d@postgrespro.ru> (raw )
In-Reply-To: <CAPmGK15bYO_A-fTcBQnmjicAn=NPyOUVygDqjByXS=ga16p4Qw@mail.gmail.com >
References: <59be194c5a409fb9fc9f2031581b8a44@postgrespro.ru >
<2fb1d9923b6995492e7b163e6cb95402@postgrespro.ru >
<DE662JHGRO2O.3KJBBG2R3IT17@gmail.com >
<efbfbcf00b6b790e8f80c13a83417a23@postgrespro.ru >
<CAFY6G8d3Yvxa_kRQA24BsJhwqfmSCv1ujiv_7b6g5isf-ZTs_Q@mail.gmail.com >
<b546d8bd5e2218f4f917fda4c93ead21@postgrespro.ru >
<DED05POJZS2W.2EZ60AOBMDDAE@gmail.com >
<bcdefce7e3db9566d0619ad2eed2a299@postgrespro.ru >
<DF0RCOYB19RC.35BW2FHPO4N4S@gmail.com >
<a16423ca928cde3650bbc219982a5ffc@postgrespro.ru >
<DF28LV8UTUOO.71JUZDH1N8F9@gmail.com >
<d5c0bfca6ee15d62df1ba72e540b4ddf@postgrespro.ru >
<DFAQTF8P9AU0.2ZELZFXOCD0XL@gmail.com >
<c264abe661f72d6627f987fc9aa4e456@postgrespro.ru >
<DFBN6K9K9IUG.23WZ7OBL8DRAQ@gmail.com >
<c0d55d3f68bfd5650fe3143d3a31c84c@postgrespro.ru >
<782a968c8e01ec6db3b2da2120adf73b@postgrespro.ru >
<554a73f7ffea8b22b3f81a4804b5fc34@postgrespro.ru >
<e13717179328abed3e189f4ca1e63087@postgrespro.ru >
<CAPpHfdu+3Eud0CBpdFT+osWiT=e=zOQUtBsx8Z5okKrqhgVAJg@mail.gmail.com >
<42a9a941-e768-4fd8-8067-12958c5a1d70@gmail.com >
<CAPpHfdsO8zYpDW==D6T5N0cJ+AzK7a_OyXJoYU1kFi=xZFTLuQ@mail.gmail.com >
<DHMH23M7UOFS.12W6PUDI1I3NH@gmail.com >
<CAPpHfdssDymMVohXw5XwdPoJ9UvSv1iN2zRcmsEhhfXXks01Zg@mail.gmail.com >
<10c97af0ce34ebdb81708b1c49ed6038@postgrespro.ru >
<fa7ae004ec282fde84c29f36e0c85979@postgrespro.ru >
<CAPpHfdtNOvgKfUFLY84dLtT=m17d=976duxUvvhYGHkLhN8GAw@mail.gmail.com >
<af6f5a17e291169f5929744acc8ddd3d@postgrespro.ru >
<CAPpHfdsAL9oXrSP25c1XYS-ew0doUST=FU3LWj6aEG0iDw2rSA@mail.gmail.com >
<e8d502937d94eb6bd18e38ff1f240a05@postgrespro.ru >
<CAPmGK14i=ss35hmf5bVM1BAS=prc8QM_nwg5rzjrjUvYE1R4Zw@mail.gmail.com >
<f207ff5cb0d48e0cc58da679095c5e04@postgrespro.ru >
<CAPmGK17tsf5zm5EeXaJipak1K4K=5hsfRD3D5HFBpu+cpb+7vw@mail.gmail.com >
<CAPpHfduW0nmtxBddkJTNheQvASr-i84hybauv+wt9d==CjCdhg@mail.gmail.com >
<CAPmGK178wmY-w+vSEQWBB4DBww=uwNbPx9s_iOgSR8-uz4kvZw@mail.gmail.com >
<CAPpHfdt0pk2hmFcj+UZe=mU4F3mSKmcxn2iNWKWtSd++1X9JZA@mail.gmail.com >
<CAPmGK15hqzy8Vm8i_C1_3bHC4ZHMcmM+rbOJgC-W=1X_gNoPKQ@mail.gmail.com >
<CAPmGK16hN2BtbOyT6cO7J9skH7qF0HxFAO68NjkNCMN8C3orCw@mail.gmail.com >
<CAPmGK15=KctsePnvKMNoYm3JOpuiZRWfWEfMs1GaBk9Rj45c5w@mail.gmail.com >
<a44e9ead6205f575da3df01b8c964e73@postgrespro.ru >
<CAPmGK15bYO_A-fTcBQnmjicAn=NPyOUVygDqjByXS=ga16p4Qw@mail.gmail.com >
Etsuro Fujita писал(а) 2026-11-01 20:44:
> Hi Alexander,
Hi.
>
> On Wed, Oct 28, 2026 at 8:37 PM Alexander Pyhalov
> <a.pyhalov@postgrespro.ru> wrote:
>> I've looked through the updated patch.
>> It looks good, but I've found some misprints in comments:
>>
>> 1) postgres_fdw.c - you have a typo here
>
> [snip]
>
>> s/csse/case/
>
> Fixed. I also tweaked the comments a bit further.
>
/*
* This is the case when an asynchronous request made by
the
- * requestor for another subplan is currently
in-progress. Skip the
- * configuration even in the case where the requestor is
MergeAppend;
- * in which csse we just defer the configuration until
the preceding
- * request(s) are processed by
ExecMergeAppendAsyncDrain.
+ * requestor for a preceding subplan is currently
in-progress. Skip
+ * the configuration even in the MergeAppend case; in
that case just
+ * difer the configuration until the caller processes
the request(s)
+ * for the preceding subplan(s).
*/
s/difer/defer/
>> 3) One more issue is that we change current semantics of
>> ForeignAsyncConfigureWait - out-of-tree FDWs
>> now should handle MergeAppendState in addition to AppendState as
>> requestor. Should we at least document it?
>
> Agreed. I updated fdwhandler.sgml to incorporate that perspective. I
> updated postgres-fdw.sgml and src/backend/executor/README as well.
>
Looks good to me.
> I also adjusted regression test cases and their comments. While doing
> that, I noticed these test cases:
>
> +-- Test async Merge Append rescan when a LIMIT stops the scan before
> every
> +-- async subplan's request has completed, so on the next rescan (for
> the
> +-- next outer row) some subplan's request is still in flight.
> +EXPLAIN (VERBOSE, COSTS OFF)
> +SELECT g.i, s.b FROM generate_series(1, 3) g(i),
> + LATERAL (SELECT b FROM async_pt WHERE a > g.i ORDER BY b LIMIT 1) s;
>
> +-- Test async Merge Append rescan when a LIMIT stops the scan before
> an
> +-- async subplan's already-delivered result was ever consumed (because
> it
> +-- didn't win the race to the heap top). The next rescan (for the
> next
> +-- outer row) must not mistake that leftover result for a fresh one;
> the
> +-- expected result increases each row, so a stale reused value would
> show
> +-- up as the first row's value repeating instead.
> +EXPLAIN (VERBOSE, COSTS OFF)
> +SELECT g.i, s.b FROM generate_series(1, 3) g(i),
> + LATERAL (SELECT b FROM async_pt WHERE a > g.i AND b > g.i * 100
> ORDER BY b LIMIT 1) s;
>
> These are testing the old implementation? There are two more test
> cases for testing the rescan, which seems to me good enough, so I just
> removed them (and adjusted the comments for the remaining test cases).
> Maybe I'm missing something, though.
>
Firstly I've confused these tests with another one:
+-- Test ExecMergeAppendAsyncReset code path that asserts async requests
+-- made for subplans aren't pending anymore at ReScan
+-- SubPlan case
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT
+ ARRAY(SELECT f.i FROM (SELECT b + g.i FROM async_pt WHERE a >
g.i ORDER BY b) f(i) ORDER BY f.i LIMIT 10)
+FROM generate_series(1, 3) g(i);
The reason is obvious - comment is misleading.
In the previous version of the patch it was
-- Test async Merge Append rescan
Now it states
-- Test ExecMergeAppendAsyncReset code path that asserts async requests
-- made for subplans aren't pending anymore at ReScan
-- SubPlan case
The original intention was not this, it was to show that valid async
subplans are recalculated after rescan (and identification of new valid
asyncplans).
Likely, the comment has to be fixed.
And yes, as now we wait for all async requests to deliver results, it
seems we can't have "old" results from previous runs while looking at
node->ms_slots, corresponding
to async plans. So, I suppose that tests that you removed, can be safely
removed.
--
Best regards,
Alexander Pyhalov,
Postgres Professional
view thread (56+ messages)
Message-ID: <0bf760a18b139ece9f447c2082499e6d@postgrespro.ru>
Permalink: ../0bf760a18b139ece9f447c2082499e6d@postgrespro.ru/
Also on: postgresql.org/message-id/0bf760a18b139ece9f447c2082499e6d@postgrespro.ru
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.pyhalov@postgrespro.ru, etsuro.fujita@gmail.com, aekorotkov@gmail.com, matheusssilv97@gmail.com
Subject: Re: Asynchronous MergeAppend
In-Reply-To: <0bf760a18b139ece9f447c2082499e6d@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox