public inbox for [email protected]  

help / color / mirror / Atom feed

From: Bossart, Nathan <[email protected]>
To: Justin Pryzby <[email protected]>
To: Robert Haas <[email protected]>
Cc: Bharath Rupireddy <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
Date: Mon, 13 Dec 2021 18:30:43 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<CALj2ACXYDPwhR6BkQuKTaJjO-y4i8kQCQxydTcBZy96UjN6FqA@mail.gmail.com>
	<[email protected]>
	<CALj2ACUPSG4aMqCEO9L+xSyfLYaL88E377rqV_Jsc7P82+U2xA@mail.gmail.com>
	<[email protected]>
	<CALj2ACUJ8LGwvc=ZzMrW7X6O5nWA11AqPHFjUdDL66kv3qQgpQ@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<CA+Tgmoag+stJPU9Vxoyaq6gzZGPLQ9C+edZcKb4PgW-yTB8LaA@mail.gmail.com>
	<[email protected]>

On 12/13/21, 9:20 AM, "Justin Pryzby" <[email protected]> wrote:
> On Mon, Dec 13, 2021 at 08:53:37AM -0500, Robert Haas wrote:
>> Another issue is that we don't want to increase the number of
>> processes without bound. Processes use memory and CPU resources and if
>> we run too many of them it becomes a burden on the system. Low-end
>> systems may not have too many resources in total, and high-end systems
>> can struggle to fit demanding workloads within the resources that they
>> have. Maybe it would be cheaper to do more things at once if we were
>> using threads rather than processes, but that day still seems fairly
>> far off.
>
> Maybe that's an argument that this should be a dynamic background worker
> instead of an auxilliary process.  Then maybe it would be controlled by
> max_parallel_maintenance_workers (or something similar).  The checkpointer
> would need to do these tasks itself if parallel workers were disabled or
> couldn't be launched.

I think this is an interesting idea.  I dislike the prospect of having
two code paths for all this stuff, but if it addresses the concerns
about resource usage, maybe it's worth it.

Nathan

view thread (59+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: O(n) tasks cause lengthy startups and checkpoints
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox