Date: Sun, 19 Jul 2020 07:38:37 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jeff Davis <pgsql@j-davis.com>, Peter Geoghegan <pg@bowt.ie>,
	Alvaro Herrera <alvherre@2ndquadrant.com>,
	David Rowley <dgrowleyml@gmail.com>,
	Andres Freund <andres@anarazel.de>,
	Bruce Momjian <bruce@momjian.us>,
	Robert Haas <robertmhaas@gmail.com>,
	Justin Pryzby <pryzby@telsasoft.com>,
	Melanie Plageman <melanieplageman@gmail.com>,
	Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: Default setting for enable_hashagg_disk
Message-ID: <20200719113837.GF12375@tamriel.snowman.net>
References: <5cf74833a830e76cc7cc7506aa77bd6ab0c94f4c.camel@j-davis.com>
 <20200713164736.GA25726@alvherre.pgsql>
 <CAH2-WzmaJG4QYHq5LR+ZXEf1JMVcTzT9pEOeMFg0uuGXYDvgzQ@mail.gmail.com>
 <9e4a4cffe3eee0543e5b94c90821d39e03b81d29.camel@j-davis.com>
 <CAH2-Wz=qBF1QyXKdB6YNV993WpQ0wj-x4h8r9yjYUwQRCMkLjA@mail.gmail.com>
 <6d0aba397625df79bdc3ca85adc11b9e48ecb0da.camel@j-davis.com>
 <4118680.1595097043@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="IcjzwMCQyeqbz4V/"
Content-Disposition: inline
In-Reply-To: <4118680.1595097043@sss.pgh.pa.us>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--IcjzwMCQyeqbz4V/
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Jeff Davis <pgsql@j-davis.com> writes:
> > On Fri, 2020-07-17 at 18:38 -0700, Peter Geoghegan wrote:
> >> There is also the separate question of what to do about the
> >> hashagg_avoid_disk_plan GUC (this is a separate open item that
> >> requires a separate resolution). Tom leans slightly towards removing
> >> it now. Is your position about the same as before?
>=20
> > Yes, I think we should have that GUC (hashagg_avoid_disk_plan) for at
> > least one release.
>=20
> You'e being optimistic about it being possible to remove a GUC once
> we ship it.  That seems to be a hard sell most of the time.

Agreed.

> I'm honestly a bit baffled about the level of fear being expressed
> around this feature.  We have *frequently* made changes that would
> change query plans, perhaps not 100.00% for the better, and never
> before have we had this kind of bikeshedding about whether it was
> necessary to be able to turn it off.  I think the entire discussion
> is way out ahead of any field evidence that we need such a knob.
> In the absence of evidence, our default position ought to be to
> keep it simple, not to accumulate backwards-compatibility kluges.

+100

> (The only reason I'm in favor of heap_mem[_multiplier] is that it
> seems like it might be possible to use it to get *better* plans
> than before.  I do not see it as a backwards-compatibility knob.)

I still don't think a hash_mem-type thing is really the right direction
to go in, even if making a distinction between memory used for sorting
and memory used for hashing is, and I'm of the general opinion that we'd
be thinking about doing something better and more appropriate- except
for the fact that we're talking about adding this in during beta.

In other words, if we'd stop trying to shoehorn something in, which
we're doing because we're in beta, we'd very likely be talking about all
of this in a very different way and probably be contemplating something
like a query_mem that provides for an overall memory limit and which
favors memory for hashing over memory for sorting, etc.

Thanks,

Stephen

--IcjzwMCQyeqbz4V/
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJfFDC8AAoJEO1sijiDR2RV8RYQALbP1Xr0W/7udw7BkaqUtkti
EUf7yAlU5dsqZiOq1I2hM3+vswnqek0+4MNbZwG4QlMJIFm8UOSNUnKRZkODU0PM
iZy43lj8uANh1GsQhJp7dBYq0KUNCps3M2Fvev9j9B2dXd9rKfXhIHLDdW8LiQmf
ThW4xLWZwBifPBnoRNJECNBMxOPvC6Cd0QWkndunRE18Jiv/ryOHtwfo3hSbuYj6
NBl64KYtMPcBIz0ANYCwsFD+JKeShx7kOBPY12BtSzyaFmsjv0t+wpW30f4Xg7td
Hok45R8Om5cmcmbwKTm9s812tlfhO5MJ7ArXA2u6pK97XWtOrwQeRgnwSpKb15lX
1zo5JVaN1NcmeZFRBsu0kDwl40P8CIinNpAvsMx+U+6d6W7MwT/flyiYmdIU8ka2
CPq7vVLxc11N1ySivn9Zi++/F/8aX6TnCv6wob+Yh6Dz0nOF8DPryLWbPtYX0PnN
kLfTUYIkMGIqEk6XwRmSTN8cNfHKlJP9BaLuOZzuV+MTdLxDYTRs6XVUIgRsHDt6
6PyxsulqcJZaozh5hi9xMrOZt8ABvJLreGa/RaqGDcGkdfUErBcZ1c/tqP0l2iy4
Os+MU99RvPt6lT/oGiE9cVj/C+CYQKmw/RcuhxTaAOA58o3RdaYNqgTFhs+wMqpm
NVw4/9zjjj6vFoHtyS0O
=V+tr
-----END PGP SIGNATURE-----

--IcjzwMCQyeqbz4V/--