From: Nathan Bossart <nathandbossart@gmail.com>
To: Corey Huinker <corey.huinker@gmail.com>
Cc: Ilya Gladyshev <ilya.v.gladyshev@gmail.com>
Cc: Daniel Gustafsson <daniel@yesql.se>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Jeff Davis <pgsql@j-davis.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: optimizing pg_upgrade's once-in-each-database steps
Date: Sat, 10 Aug 2024 10:35:46 -0500
Message-ID: <ZreI0vpZoQfoocDq@nathan> (raw)
In-Reply-To: <ZreEh54Xr6D3Qy8k@nathan>
References: <Zp677slGLwhPhVWy@nathan>
<10c1c8dd-4685-46d4-80be-56bdfca8659a@gmail.com>
<ZqvJg_lTTTt82BfQ@nathan>
<ZqwA_qngQM25FrjK@nathan>
<60cbc507-4c36-4037-9af7-028b1c8455b5@gmail.com>
<ZrJ3bu4og_Uj_9qj@nathan>
<CADkLM=e0KxKJH0XSG6x2oGK7-W2Eet4AwwCqz5pxAG7QthX4Xw@mail.gmail.com>
<ZrYrL8-LQ2LhSPQ4@nathan>
<CADkLM=fc-wujxNtGpLSVrBAtWQgsW-8RYrMxVyQ1buJfQX9CHw@mail.gmail.com>
<ZreEh54Xr6D3Qy8k@nathan>
On Sat, Aug 10, 2024 at 10:17:27AM -0500, Nathan Bossart wrote:
> On Fri, Aug 09, 2024 at 04:06:16PM -0400, Corey Huinker wrote:
>>> Furthermore, most of the callbacks should do almost nothing for a given
>>> upgrade, and since pg_upgrade runs on the server, client/server round-trip
>>> time should be pretty low.
>>
>> To my mind, that makes pipelining make more sense, you throw out N queries,
>> most of which are trivial, and by the time you cycle back around and start
>> digesting result sets via callbacks, more of the queries have finished
>> because they were waiting on the query ahead of them in the pipeline, not
>> waiting on a callback to finish consuming its assigned result set and then
>> launching the next task query.
>
> My assumption is that the "waiting for a callback before launching the next
> query" time will typically be pretty short in practice. I could try
> measuring it...
Another option might be to combine all the queries for a task into a single
string and then send that in one PQsendQuery() call. That may be a simpler
way to eliminate the time between queries.
--
nathan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, corey.huinker@gmail.com, ilya.v.gladyshev@gmail.com, daniel@yesql.se, robertmhaas@gmail.com, pgsql@j-davis.com
Subject: Re: optimizing pg_upgrade's once-in-each-database steps
In-Reply-To: <ZreI0vpZoQfoocDq@nathan>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox