Date: Fri, 2 Apr 2021 11:07:36 -0400
From: Bruce Momjian <bruce@momjian.us>
To: Laurenz Albe <laurenz.albe@cybertec.at>
Cc: "Campbell, Lance" <lance@illinois.edu>,
	MichaelDBA <MichaelDBA@sqlexec.com>,
	SASIKUMAR Devaraj <sashikumard@yahoo.com>,
	"holger@jakobs.com" <holger@jakobs.com>,
	"pgsql-admin@lists.postgresql.org" <pgsql-admin@lists.postgresql.org>
Subject: Re: work_mem
Message-ID: <20210402150736.GD9270@momjian.us>
References: <A2AA6D08-9506-4B41-9738-4D5FF8E2E011@illinois.edu>
 <04D84259-1B75-4648-949F-78677916194E@jakobs.com>
 <66290583.2329958.1617367063840@mail.yahoo.com>
 <c1e02a03-71cd-4071-e000-967403bcab9d@sqlexec.com>
 <1654783699.2333997.1617367552583@mail.yahoo.com>
 <592e5895-8dbd-1a4f-a87c-af72ac8eea02@sqlexec.com>
 <SN6PR11MB3326EADE632BA4EBE1FA28DCDE7A9@SN6PR11MB3326.namprd11.prod.outlook.com>
 <339e1ee870881a3efaffc7ea4cddac6bc4d009cf.camel@cybertec.at>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <339e1ee870881a3efaffc7ea4cddac6bc4d009cf.camel@cybertec.at>
User-Agent: Mutt/1.10.1 (2018-07-13)
Archived-At: 
 <https://www.postgresql.org/message-id/20210402150736.GD9270%40momjian.us>
Precedence: bulk

On Fri, Apr  2, 2021 at 04:59:16PM +0200, Laurenz Albe wrote:
> On Fri, 2021-04-02 at 13:31 +0000, Campbell, Lance wrote:
> > It feels like there needs to be work_mem and work_mem_stack_size.  When work memory is
> >  needed a process “pops” a token off of a stack.  When it is done processing it “puts”
> >  the token back on the stack.  If the stack is empty then don’t allocate memory just
> >  write to disk for work_mem. 
> > 
> > This does two key things:
> > 
> > 1) It allows for a real world understanding of how much memory is really needed on a
> >  day to day basis.  You can track how often a stack is empty.  You can also look at the
> >  number of temp files to see when work exceeds the work_mem allocation.  There is no
> >  “art” to setting these values.  You can use logical analysis to make choices.
> > 
> > 2) This also prevents out of memory issues.  You are protecting yourself from extreme loads.
> 
> If I get you right, you want another memory limit per session.
> 
> I see the point, but then we wouldn't need "work_mem" any more, right?
> What is the point of limiting the memory per plan node if we have an
> overall limit?
> 
> In practice, I have never had trouble with "work_mem".  I usually follow
> my rule of thumb: max_connections * work_mem + shared_buffers < RAM
> 
> While some backend may need more, many will need less.  Only bitmaps, hashes
> and sorts are memory hungry.

This blog entry discusses how work_mem might be improved:

	https://momjian.us/main/blogs/pgblog/2018.html#December_10_2018

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  If only the physical world exists, free will is an illusion.