Message-ID: <35ef2cd73b4c901feb9d04d755da27d02ec71b2e.camel@j-davis.com>
Subject: Re: Proposal: "query_work_mem" GUC, to distribute working memory to
 the query's individual operators
From: Jeff Davis <pgsql@j-davis.com>
To: James Hunter <james.hunter.pg@gmail.com>
Cc: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: Tue, 11 Feb 2025 14:04:45 -0800
In-Reply-To: <CAJVSvF7x_DLj7-JrXvMB4_j+jzuvjG_7iXNjx5KmLBTXHPNdGA@mail.gmail.com>
References: 
	<CAJVSvF6s1LgXF6KB2Cz68sHzk+v+O_vmwEkaon=H8O9VcOr-tQ@mail.gmail.com>
	 <0beedccb92ba5a1db386bd09b44e15b20c1ac1fc.camel@j-davis.com>
	 <CAJVSvF7B+z0i=jMvvcWnq=Vh8AHZwe02BSBmOfmGjiu1Nf4_wQ@mail.gmail.com>
	 <6b134f12c727fdb3a03849deab585dea0dfb30aa.camel@j-davis.com>
	 <CAJVSvF68TGsyrvMpjmPviNvR48UYkD6AN-HLQXOu2uj3HCHHGA@mail.gmail.com>
	 <e22b9d56e56d4cecd2cc7e95a9758e09cc187c8f.camel@j-davis.com>
	 <CAJVSvF7x_DLj7-JrXvMB4_j+jzuvjG_7iXNjx5KmLBTXHPNdGA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/35ef2cd73b4c901feb9d04d755da27d02ec71b2e.camel%40j-davis.com>
Precedence: bulk

On Tue, 2025-02-11 at 10:39 -0800, James Hunter wrote:
> * The Path would store "nbytes" (=3D the optimizer's estimate of how
> much working memory a given Path will use), to allow for future
> optimizer logic to consider memory usage when choosing the best Path.
>=20
> * The Plan would store a copy of "nbytes," along with "work_mem," and
> the executor would enforce work_mem. A "(work_mem on)" option to the
> "EXPLAIN" command would display both "nbytes" and "work_mem", per
> Plan
> node.

Storing work_mem in each Plan node, and using that to enforce the
memory limit (rather than using the GUC directly), seems
uncontroversial to me. I'd suggest a standalone patch.

Storing the optimizer's estimate of the memory wanted also sounds like
a good idea. Let's pick a better name than "nbytes" though; maybe
"requested_mem" or something? This change would make it a lot easier
for an extension to adjust the per-node-work_mem, and also seems like
good infrastructure for anything we build into the planner later. I
suggest a standalone patch for this, as well.

Can you write a useful extension with just the above two core patches?

Regards,
	Jeff Davis