Subject: Re: Asynchronous Append on postgres_fdw nodes.
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
 Justin Pryzby <pryzby@telsasoft.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: 
 <CAPmGK14igABfEruPPh1X=9KnjY=LquRHQij48oATSo1vWr7yOg@mail.gmail.com>
 <CAPmGK15xtFk2gb9-2b24mFtYBVWoGkmCPFPkM9RHpKVXGLk=sw@mail.gmail.com>
 <CAPmGK178d9-+aFgYorOY4n0pZmZ55WpQ7i1JcCagovYzAh+ueg@mail.gmail.com>
 <20210331.101137.148208859959175708.horikyota.ntt@gmail.com>
 <CAPmGK17T9Swy2FHZk4AVVKNzRxVnd96PX-18DSC2jeDXWNiBoQ@mail.gmail.com>
 <CAPmGK1637W30Wx3MnrReewhafn6F_0J76mrJGoFXFnpPq4QfvA@mail.gmail.com>
 <CAPmGK17O6fMfX84wWfypLWAO4NvD4fgDhm--oyzZ97YP+TxckA@mail.gmail.com>
 <2eb662bb-105d-fc20-7412-2f027cc3ca72@postgrespro.ru>
 <CAPmGK146cxSJyfM4Xv2jGTsF0ghmPEtwA7dS5_4cKo1oF6+ipw@mail.gmail.com>
 <CAPmGK17yjZg2ajhs3erpfOKj+sg6yZU_gwvaeOV+j8g2E2bdcA@mail.gmail.com>
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Organization: Postgres Professional
Message-ID: <a38bb206-8340-9528-5ef6-37de2d5cb1a3@postgrespro.ru>
Date: Fri, 7 May 2021 15:32:47 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.10.1
MIME-Version: 1.0
In-Reply-To: 
 <CAPmGK17yjZg2ajhs3erpfOKj+sg6yZU_gwvaeOV+j8g2E2bdcA@mail.gmail.com>
Content-Type: multipart/mixed;
 boundary="------------9AAB55F860D87CE044F34A7D"
Content-Language: en-US
Archived-At: 
 <https://www.postgresql.org/message-id/a38bb206-8340-9528-5ef6-37de2d5cb1a3%40postgrespro.ru>
Precedence: bulk

This is a multi-part message in MIME format.
--------------9AAB55F860D87CE044F34A7D
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

On 6/5/21 11:45, Etsuro Fujita wrote:
> On Tue, Apr 27, 2021 at 9:31 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> The patch fixes the issue, but I don’t think it’s the right way to go,
> because it requires an extra ExecProcNode() call, which wouldn’t be
> efficient.  Also, the patch wouldn’t address another issue I noticed
> in EXPLAIN ANALYZE for async-capable nodes that the command wouldn’t
> measure the time spent in such nodes accurately.  For the case of
> async-capable node using postgres_fdw, it only measures the time spent
> in ExecProcNode() in ExecAsyncRequest()/ExecAsyncNotify(), missing the
> time spent in other things such as creating a cursor in
> ExecAsyncRequest().  :-(.  To address both issues, I’d like to propose
> the attached, in which I added instrumentation support to
> ExecAsyncRequest()/ExecAsyncConfigureWait()/ExecAsyncNotify().  I
> think this would not only address the reported issue more efficiently,
> but allow to collect timing for async-capable nodes more accurately.

Ok, I agree with the approach, but the next test case failed:

EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
SELECT * FROM (
	(SELECT * FROM f1) UNION ALL (SELECT * FROM f2)
) q1 LIMIT 100;
ERROR:  InstrUpdateTupleCount called on node not yet executed

Initialization script see in attachment.

-- 
regards,
Andrey Lepikhov
Postgres Professional

--------------9AAB55F860D87CE044F34A7D
Content-Type: application/sql; x-mac-type="0"; x-mac-creator="0";
 name="t1.sql"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="t1.sql"

RFJPUCBFWFRFTlNJT04gSUYgRVhJU1RTIHBvc3RncmVzX2ZkdyBDQVNDQURFOwpEUk9QIFRB
QkxFIElGIEVYSVNUUyBsMSxsMixsMywgcGFydCBDQVNDQURFOwogCkNSRUFURSBFWFRFTlNJ
T04gcG9zdGdyZXNfZmR3OwpDUkVBVEUgU0VSVkVSIGxvb3BiYWNrMSBGT1JFSUdOIERBVEEg
V1JBUFBFUiBwb3N0Z3Jlc19mZHcgT1BUSU9OUyAoYXN5bmNfY2FwYWJsZSAndHJ1ZScpOwpD
UkVBVEUgU0VSVkVSIGxvb3BiYWNrMiBGT1JFSUdOIERBVEEgV1JBUFBFUiBwb3N0Z3Jlc19m
ZHcgT1BUSU9OUyAoYXN5bmNfY2FwYWJsZSAndHJ1ZScpOwpDUkVBVEUgU0VSVkVSIGxvb3Bi
YWNrMyBGT1JFSUdOIERBVEEgV1JBUFBFUiBwb3N0Z3Jlc19mZHcgT1BUSU9OUyAoYXN5bmNf
Y2FwYWJsZSAndHJ1ZScpOwoKQ1JFQVRFIFVTRVIgTUFQUElORyBGT1IgUFVCTElDIFNFUlZF
UiBsb29wYmFjazE7CkNSRUFURSBVU0VSIE1BUFBJTkcgRk9SIFBVQkxJQyBTRVJWRVIgbG9v
cGJhY2syOwpDUkVBVEUgVVNFUiBNQVBQSU5HIEZPUiBQVUJMSUMgU0VSVkVSIGxvb3BiYWNr
MzsKIApDUkVBVEUgVEFCTEUgbDEoYSBpbnQpOwpDUkVBVEUgRk9SRUlHTiBUQUJMRSBmMShh
IGludCkgU0VSVkVSIGxvb3BiYWNrMSBPUFRJT05TICh0YWJsZV9uYW1lICdsMScpOwpDUkVB
VEUgVEFCTEUgbDIoYSBpbnQpOwpDUkVBVEUgRk9SRUlHTiBUQUJMRSBmMihhIGludCkgU0VS
VkVSIGxvb3BiYWNrMiBPUFRJT05TICh0YWJsZV9uYW1lICdsMicpOwpDUkVBVEUgVEFCTEUg
bDMoYSBpbnQpOwpDUkVBVEUgRk9SRUlHTiBUQUJMRSBmMyhhIGludCkgU0VSVkVSIGxvb3Bi
YWNrMyBPUFRJT05TICh0YWJsZV9uYW1lICdsMycpOwogCkNSRUFURSBUQUJMRSBwYXJ0KGEg
aW50KSBQQVJUSVRJT04gQlkgSEFTSCAoYSk7CkFMVEVSIFRBQkxFIHBhcnQgQVRUQUNIIFBB
UlRJVElPTiBmMQogICAgRk9SIFZBTFVFUyBXSVRIIChNT0RVTFVTIDMsIFJFTUFJTkRFUiAw
KTsKQUxURVIgVEFCTEUgcGFydCBBVFRBQ0ggUEFSVElUSU9OIGYyCiAgICBGT1IgVkFMVUVT
IFdJVEggKE1PRFVMVVMgMywgUkVNQUlOREVSIDEpOwpBTFRFUiBUQUJMRSBwYXJ0IEFUVEFD
SCBQQVJUSVRJT04gZjMKICAgIEZPUiBWQUxVRVMgV0lUSCAoTU9EVUxVUyAzLCBSRU1BSU5E
RVIgMik7CiAgICAgCklOU0VSVCBJTlRPIHBhcnQgU0VMRUNUICogRlJPTSBnZW5lcmF0ZV9z
ZXJpZXMoMSwxMDAwMCk7CklOU0VSVCBJTlRPIHBhcnQgU0VMRUNUICogRlJPTSBnZW5lcmF0
ZV9zZXJpZXMoMSwxMDAwMCk7CkFOQUxZWkUgbDEsbDIsbDMsZjEsZjIsZjMscGFydDsK
--------------9AAB55F860D87CE044F34A7D--