public inbox for [email protected]
help / color / mirror / Atom feedFrom: Justin Pryzby <[email protected]>
To: Robert Haas <[email protected]>
Cc: Bossart, Nathan <[email protected]>
Cc: Bharath Rupireddy <[email protected]>
Cc: [email protected]
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
Date: Mon, 13 Dec 2021 11:19:35 -0600
Message-ID: <[email protected]> (raw)
In-Reply-To: <CA+Tgmoag+stJPU9Vxoyaq6gzZGPLQ9C+edZcKb4PgW-yTB8LaA@mail.gmail.com>
References: <[email protected]>
<CALj2ACXYDPwhR6BkQuKTaJjO-y4i8kQCQxydTcBZy96UjN6FqA@mail.gmail.com>
<[email protected]>
<CALj2ACUPSG4aMqCEO9L+xSyfLYaL88E377rqV_Jsc7P82+U2xA@mail.gmail.com>
<[email protected]>
<CALj2ACUJ8LGwvc=ZzMrW7X6O5nWA11AqPHFjUdDL66kv3qQgpQ@mail.gmail.com>
<[email protected]>
<[email protected]>
<CA+Tgmoag+stJPU9Vxoyaq6gzZGPLQ9C+edZcKb4PgW-yTB8LaA@mail.gmail.com>
On Mon, Dec 13, 2021 at 08:53:37AM -0500, Robert Haas wrote:
> On Fri, Dec 10, 2021 at 2:03 PM Bossart, Nathan <[email protected]> wrote:
> > Well, I haven't had a chance to look at your patch, and my patch set
> > still only has handling for CheckPointSnapBuild() and
> > RemovePgTempFiles(), but I thought I'd share what I have anyway. I
> > split it into 5 patches:
> >
> > 0001 - Adds a new "custodian" auxiliary process that does nothing.
...
>
> I don't know whether this kind of idea is good or not.
...
>
> Another issue is that we don't want to increase the number of
> processes without bound. Processes use memory and CPU resources and if
> we run too many of them it becomes a burden on the system. Low-end
> systems may not have too many resources in total, and high-end systems
> can struggle to fit demanding workloads within the resources that they
> have. Maybe it would be cheaper to do more things at once if we were
> using threads rather than processes, but that day still seems fairly
> far off.
Maybe that's an argument that this should be a dynamic background worker
instead of an auxilliary process. Then maybe it would be controlled by
max_parallel_maintenance_workers (or something similar). The checkpointer
would need to do these tasks itself if parallel workers were disabled or
couldn't be launched.
--
Justin
view thread (59+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox