public inbox for [email protected]
help / color / mirror / Atom feedFrom: Nathan Bossart <[email protected]>
To: Robert Haas <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: Michael Paquier <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Thomas Munro <[email protected]>
Cc: Fujii Masao <[email protected]>
Cc: Postgres hackers <[email protected]>
Subject: Re: Weird failure with latches in curculio on v15
Date: Thu, 9 Feb 2023 09:23:08 -0800
Message-ID: <20230209172308.GA605342@nathanxps13> (raw)
In-Reply-To: <CA+TgmoY6xxDdZ1YJjMb27Ei7qgFcbu=__WtDX_cOd3GQ5uvxWw@mail.gmail.com>
References: <20230206004631.GA277434@nathanxps13>
<CA+TgmoYMk+NtLxU8xpAvWe_6K783d2Pr09cUpUJFRChdj_rcbQ@mail.gmail.com>
<20230208174350.GB451849@nathanxps13>
<CA+TgmobY5ALQE4U2q32YohLbJfze8CDBZ3rs2x2+ccAa+Pqw6g@mail.gmail.com>
<20230208222554.GB546776@nathanxps13>
<[email protected]>
<20230209002413.GA603595@nathanxps13>
<CA+TgmobE6rRs-tMArr733y9VucgNA5Nd9_qYgEVWjJDwHBwP-Q@mail.gmail.com>
<[email protected]>
<CA+TgmoY6xxDdZ1YJjMb27Ei7qgFcbu=__WtDX_cOd3GQ5uvxWw@mail.gmail.com>
On Thu, Feb 09, 2023 at 11:12:21AM -0500, Robert Haas wrote:
> On Thu, Feb 9, 2023 at 10:51 AM Tom Lane <[email protected]> wrote:
>> If we think we need primitives to let the archiver hooks get all
>> the pending files, or whatever, by all means add those. But don't
>> cede fundamental control of the archiver. The hooks need to be
>> decoration on a framework we provide, not the framework themselves.
>
> I don't quite see how you can make asynchronous and parallel archiving
> work if the archiver process only calls into the archive module at
> times that it chooses. That would mean that the module has to return
> control to the archiver when it's in the middle of archiving one or
> more files -- and then I don't see how it can get control back at the
> appropriate time. Do you have a thought about that?
I've been thinking about this, actually. I'm wondering if we could provide
a list of files to the archiving callback (configurable via a variable in
ArchiveModuleState), and then have the callback return a list of files that
are archived. (Or maybe we just put the list of files that need archiving
in ArchiveModuleState.) The returned list could include files that were
sent to the callback previously. The archive module would be responsible
for creating background worker(s) (if desired), dispatching files
to-be-archived to its background worker(s), and gathering the list of
archived files to return.
This is admittedly half-formed, but I'm tempted to hack something together
quickly to see whether it might be viable.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
view thread (25+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Weird failure with latches in curculio on v15
In-Reply-To: <20230209172308.GA605342@nathanxps13>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox