From: Stephen Frost <sfrost@snowman.net>
To: Marco Fortina <marco_fortina@hotmail.it>
Cc: pgsql-general@postgresql.org <pgsql-general@postgresql.org>
Subject: Re: R: Proposal: Shared Work Mem Area
Date: Thu, 6 Apr 2023 10:34:30 -0400
Message-ID: <ZC7Ydv5PxdcfjmPZ@tamriel.snowman.net> (raw)
In-Reply-To: <PA4PR10MB5516E4AF81ADE3E4BA602BC9F7919@PA4PR10MB5516.EURPRD10.PROD.OUTLOOK.COM>
References: <PA4PR10MB5516E4AF81ADE3E4BA602BC9F7919@PA4PR10MB5516.EURPRD10.PROD.OUTLOOK.COM>
Greetings,
Please don't top-post on the PG mailing lists, it makes it harder to
follow the discussion.
* Marco Fortina (marco_fortina@hotmail.it) wrote:
> https://commitfest.postgresql.org/42/3867/ is not exacly what I proposed as new feature to developers.
I understood what you were proposing.
> If I'm not wrong, almost all main memory areas have a fixed size:
>
> shared_buffers
> effective_cache_size
> wal_buffers
>
> Instead, work_mem is per-process dynamically allocated up to defined size limit.
That's not how work_mem works actually. It's a per-node amount and it's
not a per-process overall limit, nor is it really a hard limit though
some nodes will do their best to respect the amount configured.
> What I suggested is to replace work_mem from per-process allocation to global and fixed size allocation (see pga_aggregate_target on Oracle) and shared to worker processes.
I understood the suggestion and it's a lot of work for an unclear gain.
You noted that having it be pulled from a single area would allow
administrators to configure an overall memory usage limit- but that's
not the only way to do that and there's an existing effort to do exactly
that already underway that's a lot simpler than what you're proposing.
While there might be other advantages to having a shared memory segment
be used for work_mem, you've not outlined any.
> Let's assume the new parameter name is worker_mem_area and this was set to 8GB: with my proposal method each worker process do not use it's own dedicated work_mem but the shared one.
I understand your suggestion, but making such a large change just to
make it isn't sensible, there should be reasoning behind why that's
better than what we're doing already or proposing to do.
> In this way each worker is also able to peek free pages from the worker_mem_area if needed.
This can be done with the existing approach and doesn't require a shared
memory segment for work_mem. We are pretty far from having an actual
acceptance system for queries though but I do agree that would be a
useful thing to work towards. I don't know that it requires work_mem
being in shared memory though.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 832B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: sfrost@snowman.net, marco_fortina@hotmail.it
Subject: Re: R: Proposal: Shared Work Mem Area
In-Reply-To: <ZC7Ydv5PxdcfjmPZ@tamriel.snowman.net>
References: <PA4PR10MB5516E4AF81ADE3E4BA602BC9F7919@PA4PR10MB5516.EURPRD10.PROD.OUTLOOK.COM> <ZC7Ydv5PxdcfjmPZ@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox