From: Justin Pryzby <pryzby@telsasoft.com>
To: Stephen Frost <sfrost@snowman.net>
Cc: Peter Geoghegan <pg@bowt.ie>
Cc: Jeff Davis <pgsql@j-davis.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: David Rowley <dgrowleyml@gmail.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Default setting for enable_hashagg_disk
Date: Thu, 9 Jul 2020 19:18:52 -0500
Message-ID: <20200710001852.GC26220@telsasoft.com> (raw)
In-Reply-To: <20200709225840.GE12375@tamriel.snowman.net>
References: <85F9F4F0-918A-457F-BC9E-F2AC2EFC1F86@anarazel.de>
<20200625225853.GA11137@alvherre.pgsql>
<20200708140037.GI3125@tamriel.snowman.net>
<99b047386525b87df1e611335c2e57a6cdef346b.camel@j-davis.com>
<20200709140330.GO3125@tamriel.snowman.net>
<CAH2-WzmB04dxgRgjDGvDbugEFfHxMi2pyuJqsYr30x1RAEqSXg@mail.gmail.com>
<20200709225840.GE12375@tamriel.snowman.net>
On Thu, Jul 09, 2020 at 06:58:40PM -0400, Stephen Frost wrote:
> * Peter Geoghegan (pg@bowt.ie) wrote:
> > On Thu, Jul 9, 2020 at 7:03 AM Stephen Frost <sfrost@snowman.net> wrote:
> > It makes more sense than simply ignoring what our users will see as a
> > simple regression. (Though I still lean towards fixing the problem by
> > introducing hash_mem, which at least tries to fix the problem head
> > on.)
>
> The presumption that this will always end up resulting in a regression
> really doesn't seem sensible to me.
Nobody said "always" - we're concerned about a fraction of workloads which
regress, badly affecting only only a small fraction of users.
Maybe pretend that Jeff implemented something called CashAgg, which does
everything HashAgg does but implemented from scratch. Users would be able to
tune it or disable it, and we could talk about removing HashAgg for the next 3
years. But instead we decide to remove HashAgg right now since it's redundant.
That's a bad way to transition from an old behavior to a new one. It's scary
because it imposes a burden, rather than offering a new option without also
taking away the old one.
> > That's not the only justification. The other justification is that
> > it's generally reasonable to prefer giving hash aggregate more memory.
>
> Sure, and it's generally reasonably to prefer giving Sorts more memory
> too... as long as you've got it available.
I believe he meant:
"it's generally reasonable to prefer giving hash aggregate more memory THAN OTHER NODES"
Referencing:
https://www.postgresql.org/message-id/CAH2-Wz=YEMOeXdAPwZo7uriR5KPsf_RGuMHvk3HvLDVksdrwHg@mail.gmail...https://www.postgresql.org/message-id/CAH2-Wznd_wL+Q3sUjLN3o5F6Q5AvHSTYOozPAei2QfuYDSd4fw@mail.gmail...https://www.postgresql.org/message-id/CAH2-Wz=osB4oi_nH8MnosYhVVSNOm5q3=exGe-b3q6gWOgf98w@mail.gmail...
Peter's patch and David's competing proposal aim to provide a kind of "soft"
transition, where the old behavior is still possible (whether default or not).
David's proposal (enable_hashagg=neverspill/soft) [0] isn't implemented, but
would allow returning *exactly* to the pre-13 behavior (modulo other planner
changes). That's a bit of a kludge and we'd hope to eventually remove the
neverspill/soft stuff, returning enable_* to boolean.
Whereas Peter's hash_mem patch allows returning to the pre-13 behavior if the
mem required for hashagg is between work_mem and hash_mem; but, by definition,
not if the required mem exceeds hash_mem, in which case it would spill to disk.
This has the advantage that it's independently useful and not a transitional
kludge.
--
Justin
[0] Bruce proposed some "naming things" amendments to David's idea:
https://www.postgresql.org/message-id/CAApHDvqFZikXhAGW=UKZKq1_FzHy+XzmUzAJiNj6RWyTHH4UfA@mail.gmail...https://www.postgresql.org/message-id/20200625171756.GB12486@momjian.us
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, sfrost@snowman.net, pg@bowt.ie, pgsql@j-davis.com, alvherre@2ndquadrant.com, andres@anarazel.de, bruce@momjian.us, robertmhaas@gmail.com, dgrowleyml@gmail.com, melanieplageman@gmail.com, tomas.vondra@2ndquadrant.com
Subject: Re: Default setting for enable_hashagg_disk
In-Reply-To: <20200710001852.GC26220@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox