public inbox for [email protected]  

help / color / mirror / Atom feed

From: Jehan-Guillaume de Rorthais <[email protected]>
To: Matthias van de Meent <[email protected]>
Cc: Konstantin Knizhnik <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: OOM in hash join
Date: Fri, 14 Apr 2023 13:43:21 +0200
Message-ID: <20230414134321.46fd18d3@karst> (raw)
In-Reply-To: <CAEze2Wi1F3GuPapyXe54k29JqzbzQ5Mc3mng=nFSOYyonQ+NTw@mail.gmail.com>
References: <[email protected]>
	<CAEze2Wi1F3GuPapyXe54k29JqzbzQ5Mc3mng=nFSOYyonQ+NTw@mail.gmail.com>

On Fri, 14 Apr 2023 13:21:05 +0200
Matthias van de Meent <[email protected]> wrote:

> On Fri, 14 Apr 2023 at 12:59, Konstantin Knizhnik <[email protected]> wrote:
> >
> > Hi hackers,
> >
> > Too small value of work_mem cause memory overflow in parallel hash join
> > because of too much number batches.
> > There is the plan:  
> 
> [...]
> 
> > There is still some gap between size reported by memory context sump and
> > actual size of backend.
> > But is seems to be obvious, that trying to fit in work_mem
> > sharedtuplestore creates so much batches, that  them consume much more
> > memory than work_mem.

Indeed. The memory consumed by batches is not accounted and the consumption
reported in explain analyze is wrong.

Would you be able to test the latest patchset posted [1] ? This does not fix
the work_mem overflow, but it helps to keep the number of batches
balanced and acceptable. Any feedback, comment or review would be useful.

[1] https://www.postgresql.org/message-id/flat/20230408020119.32a0841b%40karst#616c1f41fcc10e8f89d41e8e5...

Regards,

view thread (3+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: OOM in hash join
  In-Reply-To: <20230414134321.46fd18d3@karst>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox