From: Tom Lane <tgl@sss.pgh.pa.us>
To: Stephen Frost <sfrost@snowman.net>
cc: Jeff Davis <pgsql@j-davis.com>, Peter Geoghegan <pg@bowt.ie>,
        Alvaro Herrera <alvherre@2ndquadrant.com>,
        David Rowley <dgrowleyml@gmail.com>,
        Andres Freund <andres@anarazel.de>, Bruce Momjian <bruce@momjian.us>,
        Robert Haas <robertmhaas@gmail.com>,
        Justin Pryzby <pryzby@telsasoft.com>,
        Melanie Plageman <melanieplageman@gmail.com>,
        Tomas Vondra <tomas.vondra@2ndquadrant.com>,
        "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: Default setting for enable_hashagg_disk
In-reply-to: <20200719113837.GF12375@tamriel.snowman.net>
References: <5cf74833a830e76cc7cc7506aa77bd6ab0c94f4c.camel@j-davis.com>
 <20200713164736.GA25726@alvherre.pgsql>
 <CAH2-WzmaJG4QYHq5LR+ZXEf1JMVcTzT9pEOeMFg0uuGXYDvgzQ@mail.gmail.com>
 <9e4a4cffe3eee0543e5b94c90821d39e03b81d29.camel@j-davis.com>
 <CAH2-Wz=qBF1QyXKdB6YNV993WpQ0wj-x4h8r9yjYUwQRCMkLjA@mail.gmail.com>
 <6d0aba397625df79bdc3ca85adc11b9e48ecb0da.camel@j-davis.com>
 <4118680.1595097043@sss.pgh.pa.us>
 <20200719113837.GF12375@tamriel.snowman.net>
Comments: In-reply-to Stephen Frost <sfrost@snowman.net>
	message dated "Sun, 19 Jul 2020 07:38:37 -0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <67513.1595169829.1@sss.pgh.pa.us>
Date: Sun, 19 Jul 2020 10:43:49 -0400
Message-ID: <67514.1595169829@sss.pgh.pa.us>
Precedence: bulk

Stephen Frost <sfrost@snowman.net> writes:
> In other words, if we'd stop trying to shoehorn something in, which
> we're doing because we're in beta, we'd very likely be talking about all
> of this in a very different way and probably be contemplating something
> like a query_mem that provides for an overall memory limit and which
> favors memory for hashing over memory for sorting, etc.

Even if we were at the start of the dev cycle rather than its end,
I'm not sure I agree.  Yes, replacing work_mem with some more-holistic
approach would be great.  But that's a research project, one that
we can't be sure will yield fruit on any particular schedule.  (Seeing
that we've understood this to be a problem for *decades*, I would tend
to bet on a longer not shorter time frame for a solution.)

I think that if we are worried about hashagg-spill behavior in the near
term, we have to have some fix that's not conditional on solving that
very large problem.  The only other practical alternative is "do
nothing for v13", and I agree with the camp that doesn't like that.

			regards, tom lane