MIME-Version: 1.0
References: <CAJVSvF6s1LgXF6KB2Cz68sHzk+v+O_vmwEkaon=H8O9VcOr-tQ@mail.gmail.com>
 <0beedccb92ba5a1db386bd09b44e15b20c1ac1fc.camel@j-davis.com>
 <CAJVSvF7B+z0i=jMvvcWnq=Vh8AHZwe02BSBmOfmGjiu1Nf4_wQ@mail.gmail.com>
 <6b134f12c727fdb3a03849deab585dea0dfb30aa.camel@j-davis.com>
 <CAJVSvF68TGsyrvMpjmPviNvR48UYkD6AN-HLQXOu2uj3HCHHGA@mail.gmail.com> <e22b9d56e56d4cecd2cc7e95a9758e09cc187c8f.camel@j-davis.com>
In-Reply-To: <e22b9d56e56d4cecd2cc7e95a9758e09cc187c8f.camel@j-davis.com>
From: James Hunter <james.hunter.pg@gmail.com>
Date: Tue, 11 Feb 2025 10:39:51 -0800
Message-ID: <CAJVSvF7x_DLj7-JrXvMB4_j+jzuvjG_7iXNjx5KmLBTXHPNdGA@mail.gmail.com>
Subject: Re: Proposal: "query_work_mem" GUC, to distribute working memory to
 the query's individual operators
To: Jeff Davis <pgsql@j-davis.com>
Cc: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAJVSvF7x_DLj7-JrXvMB4_j%2BjzuvjG_7iXNjx5KmLBTXHPNdGA%40mail.gmail.com>
Precedence: bulk

On Tue, Feb 11, 2025 at 10:00=E2=80=AFAM Jeff Davis <pgsql@j-davis.com> wro=
te:
>
> On Mon, 2025-02-10 at 19:09 -0800, James Hunter wrote:
> > I think it makes sense to split the work into two parts: one part
> > that
> > improves SQL execution, and a second part that improves the
> > optimizer,
> > to reflect the improvements to execution.
>
> I like the idea to store the value of work_mem in the
> path/plan/executor nodes, and use that at execution time rather than
> the GUC directly.
>
> IIUC, that would allow an extension to do what you want, right? A
> planner hook could just walk the tree and edit those values for
> individual nodes, and the executor would enforce them.

Yes, exactly!

* The Path would store "nbytes" (=3D the optimizer's estimate of how
much working memory a given Path will use), to allow for future
optimizer logic to consider memory usage when choosing the best Path.

* The Plan would store a copy of "nbytes," along with "work_mem," and
the executor would enforce work_mem. A "(work_mem on)" option to the
"EXPLAIN" command would display both "nbytes" and "work_mem", per Plan
node.

* Either built-in logic or an extensibility hook would set "work_mem"
on each individual Plan node, based on whatever heuristic or rule it
chooses.

Right now, my prototype sets "work_mem" inside ExecInitNode().

Thanks,
James