From: Stephen Frost <sfrost@snowman.net>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Jeff Davis <pgsql@j-davis.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: David Rowley <dgrowleyml@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: Default setting for enable_hashagg_disk
Date: Wed, 8 Jul 2020 10:00:37 -0400
Message-ID: <20200708140037.GI3125@tamriel.snowman.net> (raw)
In-Reply-To: <20200625225853.GA11137@alvherre.pgsql>
References: <85F9F4F0-918A-457F-BC9E-F2AC2EFC1F86@anarazel.de>
<20200625225853.GA11137@alvherre.pgsql>
Greetings,
* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:
> On 2020-Jun-25, Andres Freund wrote:
>
> > >My point here is that maybe we don't need to offer a GUC to explicitly
> > >turn spilling off; it seems sufficient to let users change work_mem so
> > >that spilling will naturally not occur. Why do we need more?
> >
> > That's not really a useful escape hatch, because I'll often lead to
> > other nodes using more memory.
>
> Ah -- other nodes in the same query -- you're right, that's not good.
It's exactly how the system has been operating for, basically, forever,
for everything. Yes, it'd be good to have a way to manage the
overall amount of memory that a query is allowed to use but that's a
huge change and inventing some new 'hash_mem' or some such GUC doesn't
strike me as a move in the right direction- are we going to have
sort_mem next? What if having one large hash table for aggregation
would be good, but having the other aggregate use a lot of memory would
run the system out of memory? Yes, we need to do better, but inventing
new node_mem GUCs isn't the direction to go in.
That HashAgg previously didn't care that it was going wayyyyy over
work_mem was, if anything, a bug. Inventing new GUCs late in the
cycle like this under duress seems like a *really* bad idea. Yes,
people are going to have to adjust work_mem if they want these queries
to continue using a ton of memory to run when the planner didn't think
it'd actually take that much memory- but then, in lots of the kinds of
cases that I think you're worrying about, the stats aren't actually that
far off and people did increase work_mem to get the HashAgg plan in the
first place.
I'm also in support of having enable_hashagg_disk set to true as the
default, just like all of the other enable_*'s.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: sfrost@snowman.net, alvherre@2ndquadrant.com, andres@anarazel.de, bruce@momjian.us, pgsql@j-davis.com, robertmhaas@gmail.com, dgrowleyml@gmail.com, pryzby@telsasoft.com, melanieplageman@gmail.com, tomas.vondra@2ndquadrant.com
Subject: Re: Default setting for enable_hashagg_disk
In-Reply-To: <20200708140037.GI3125@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox