From: Nathan Bossart <nathandbossart@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Fujii Masao <fujii@postgresql.org>
Cc: Postgres hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Weird failure with latches in curculio on v15
Date: Sun, 5 Feb 2023 15:57:47 -0800
Message-ID: <20230205235747.GA275913@nathanxps13> (raw)
In-Reply-To: <20230205230157.45kl5gqryupzdoyb@alap3.anarazel.de>
References: <1369666.1675264346@sss.pgh.pa.us>
<20230201165801.33ydbxvjdbomjqa7@alap3.anarazel.de>
<20230201175806.GA3199959@nathanxps13>
<20230201223555.GA3721373@nathanxps13>
<Y9sam108o4mxZFiS@paquier.xyz>
<20230204113029.xlcqrbxhp6lerrnc@alap3.anarazel.de>
<20230204180354.GA258107@nathanxps13>
<Y979NR3U5VnWrTwB@paquier.xyz>
<20230205221938.GA274245@nathanxps13>
<20230205230157.45kl5gqryupzdoyb@alap3.anarazel.de>
On Sun, Feb 05, 2023 at 03:01:57PM -0800, Andres Freund wrote:
> I think at the very least you'd want to have a separate callback for
> restoring segments than for restoring other files. But more likely a
> separate callback for each type of file to be restored.
>
> For the timeline history case an parameter indicating that we don't want
> to restore the file, just to see if there's a conflict, would make
> sense.
That seems reasonable.
> For the segment files, we'd likely need a parameter to indicate whether
> the restore is random or not.
Wouldn't this approach still require each module to handle restoring ahead
of time? I agree that the shell overhead isn't the main performance issue,
but it's unclear to me how much of this should be baked into PostgreSQL. I
mean, we could introduce a GUC that tells us how far ahead to restore and
have a background worker (or multiple background workers) asynchronously
pull files into a staging directory via the callbacks. Is that the sort of
scope you are envisioning?
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, andres@anarazel.de, michael@paquier.xyz, tgl@sss.pgh.pa.us, thomas.munro@gmail.com, fujii@postgresql.org, pgsql-hackers@lists.postgresql.org
Subject: Re: Weird failure with latches in curculio on v15
In-Reply-To: <20230205235747.GA275913@nathanxps13>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox