pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Asynchronous Append on postgres_fdw nodes.
Date: Thu, 3 Jun 2021 20:33:56 +0500
Message-ID: <520f3426-466c-0aa2-1bae-6188466203db@postgrespro.ru> (raw)
In-Reply-To: <CAPmGK14h4dYJ=O3xKdFPYnK_pG9MwmnmOHwyryvFJW9v1th-aw@mail.gmail.com>
References: <CAPmGK14igABfEruPPh1X=9KnjY=LquRHQij48oATSo1vWr7yOg@mail.gmail.com>
	<20210331.101137.148208859959175708.horikyota.ntt@gmail.com>
	<CAPmGK17T9Swy2FHZk4AVVKNzRxVnd96PX-18DSC2jeDXWNiBoQ@mail.gmail.com>
	<CAPmGK1637W30Wx3MnrReewhafn6F_0J76mrJGoFXFnpPq4QfvA@mail.gmail.com>
	<CAPmGK17O6fMfX84wWfypLWAO4NvD4fgDhm--oyzZ97YP+TxckA@mail.gmail.com>
	<2eb662bb-105d-fc20-7412-2f027cc3ca72@postgrespro.ru>
	<CAPmGK146cxSJyfM4Xv2jGTsF0ghmPEtwA7dS5_4cKo1oF6+ipw@mail.gmail.com>
	<CAPmGK17yjZg2ajhs3erpfOKj+sg6yZU_gwvaeOV+j8g2E2bdcA@mail.gmail.com>
	<a38bb206-8340-9528-5ef6-37de2d5cb1a3@postgrespro.ru>
	<CAPmGK16R=U2ok=H_QnibLS297_vmt68b6bJ8OYJzQ0DxMo+gyw@mail.gmail.com>
	<2080193d-da8d-4132-f48c-8e29be8fcffc@postgrespro.ru>
	<CAPmGK15CyEmstJyzkGsT_1OmArJwD6vQguBEwiz8oYx4zz6ssA@mail.gmail.com>
	<58fa2aa5-07f5-80b5-59a1-fec8a349fee7@postgrespro.ru>
	<CAPmGK15h+GJxFA6nKWQ2RSO8TUgNWn_3+h4DJzvvW==A_Wm02g@mail.gmail.com>
	<CAPmGK14h4dYJ=O3xKdFPYnK_pG9MwmnmOHwyryvFJW9v1th-aw@mail.gmail.com>

On 3/6/21 14:49, Etsuro Fujita wrote:
> On Tue, May 11, 2021 at 6:55 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
>> On Tue, May 11, 2021 at 6:27 PM Andrey Lepikhov
>> <a.lepikhov@postgrespro.ru> wrote:
>>> On 11/5/21 12:24, Etsuro Fujita wrote:
>>
>>>>>       ->  Append (actual rows=3000 loops=1)
>>>>>             ->  Async Foreign Scan on f1 (actual rows=0 loops=1)
>>>>>             ->  Async Foreign Scan on f2 (actual rows=0 loops=1)
>>>>>             ->  Foreign Scan on f3 (actual rows=3000 loops=1)
>>>>>
>>>>> Here we give preference to the synchronous scan. Why?
>>>>
>>>> This would be expected behavior, and the reason is avoid performance
>>>> degradation; you might think it would be better to execute the async
>>>> Foreign Scan nodes more aggressively, but it would require
>>>> waiting/polling for file descriptor events many times, which is
>>>> expensive and might cause performance degradation.  I think there is
>>>> room for improvement, though.
>>> Yes, I agree with you. Maybe you can add note in documentation on
>>> async_capable, for example:
>>> "... Synchronous and asynchronous scanning strategies can be mixed by
>>> optimizer in one scan plan of a partitioned table or an 'UNION ALL'
>>> command. For performance reasons, synchronous scans executes before the
>>> first of async scan. ..."
>>
>> +1  But I think this is an independent issue, so I think it would be
>> better to address the issue separately.
> 
> I think that since postgres-fdw.sgml would be for users rather than
> developers, unlike fdwhandler.sgml, it would be better to explain this
> more in a not-too-technical way.  So how about something like this?
> 
> Asynchronous execution is applied even when an Append node contains
> subplan(s) executed synchronously as well as subplan(s) executed
> asynchronously.  In that case, if the asynchronous subplans are ones
> executed using postgres_fdw, tuples from the asynchronous subplans are
> not returned until after at least one synchronous subplan returns all
> tuples, as that subplan is executed while the asynchronous subplans
> are waiting for the results of queries sent to foreign servers.  This
> behavior might change in a future release.
Good, this text is clear for me.

-- 
regards,
Andrey Lepikhov
Postgres Professional





view thread (130+ messages)  latest in thread

Message-ID: <520f3426-466c-0aa2-1bae-6188466203db@postgrespro.ru>
Permalink:  ../520f3426-466c-0aa2-1bae-6188466203db@postgrespro.ru/
Also on:    postgresql.org/message-id/520f3426-466c-0aa2-1bae-6188466203db@postgrespro.ru

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: a.lepikhov@postgrespro.ru, etsuro.fujita@gmail.com, horikyota.ntt@gmail.com, pryzby@telsasoft.com, pgsql-hackers@lists.postgresql.org
  Subject: Re: Asynchronous Append on postgres_fdw nodes.
  In-Reply-To: <520f3426-466c-0aa2-1bae-6188466203db@postgrespro.ru>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox