MIME-Version: 1.0
References: 
 <CAPmGK17dAZCXvwnfpr1eTfknTGdt=hYTV9405Gt5SqPOX8K84w@mail.gmail.com>
 <65af8fb4-4c45-9cc3-c6ec-80f3ac488e07@oss.nttdata.com>
 <CAPmGK17XczZmzU=0mhk2_oG9Omo1p-uZCNdZAh_3vuC6+YboiA@mail.gmail.com>
 <95066202-c7e6-e4be-05cc-bdceb45b68c7@oss.nttdata.com>
 <TYAPR01MB58668C69BA6CC0FEA79D7336F5949@TYAPR01MB5866.jpnprd01.prod.outlook.com>
In-Reply-To: 
 <TYAPR01MB58668C69BA6CC0FEA79D7336F5949@TYAPR01MB5866.jpnprd01.prod.outlook.com>
From: Etsuro Fujita <etsuro.fujita@gmail.com>
Date: Wed, 17 Nov 2021 13:00:04 +0900
Message-ID: 
 <CAPmGK16eZm3BegNzTM6cCYNqq1+vU31hqW76DW1Uq7ToFc-_Xw@mail.gmail.com>
Subject: Re: postgres_fdw: commit remote (sub)transactions in parallel during
 pre-commit
To: "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
	Fujii Masao <masao.fujii@oss.nttdata.com>
Content-Type: text/plain; charset="UTF-8"
Archived-At: 
 <https://www.postgresql.org/message-id/CAPmGK16eZm3BegNzTM6cCYNqq1%2BvU31hqW76DW1Uq7ToFc-_Xw%40mail.gmail.com>
Precedence: bulk

Kuroda-san,

On Thu, Nov 11, 2021 at 11:27 AM kuroda.hayato@fujitsu.com
<kuroda.hayato@fujitsu.com> wrote:
> I love your proposal because it will remove a bottleneck
> for PostgreSQL build-in sharding.
>
> I read your patch briefly and I think basically it's good.

Great!  Thanks for reviewing!

> Currently I have only one comment.
>
> In your patch, postgres_fdw sends a COMMIT command to all entries in the hash table
> and waits for the result without a timeout from the first entry.
> I think this specification is good because it's very simple,
> but if a COMMIT for a particular foreign server could take some time,
> I thought it might be more efficient to stop waiting for results and look at the next entry.
> This is how it works. First, we define a function similar to pgfdw_get_result()
> so that we can specify the timeout time as an argument to WaitLatchOrSocket().
> Then change the function called by do_sql_command_end () to the new one,
> and change the callback function to skip if the result has not yet arrived
>
> How is it? Is it an unnecessary assumption that COMMIT takes time? Or is this the next step?
> I will put a PoC if needed.

Hmm, I'm not sure the cost-effectiveness of this optimization is
really high, because if the timeout expired, it means that something
unusual would have happened, and that it would take a long time for
the COMMIT command to complete (or abort at worst).  So even if we
processed the rest of the entries while waiting for the command
result, we cannot reduce the total time very much.  Maybe I'm missing
something, though.

Best regards,
Etsuro Fujita