Date: Fri, 10 Jul 2020 10:34:15 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Peter Geoghegan <pg@bowt.ie>,
	Alvaro Herrera <alvherre@2ndquadrant.com>,
	Andres Freund <andres@anarazel.de>,
	Bruce Momjian <bruce@momjian.us>,
	Robert Haas <robertmhaas@gmail.com>,
	David Rowley <dgrowleyml@gmail.com>,
	Melanie Plageman <melanieplageman@gmail.com>,
	Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	pgsql-hackers@postgresql.org
Subject: Re: Default setting for enable_hashagg_disk
Message-ID: <20200710143415.GJ12375@tamriel.snowman.net>
References: <85F9F4F0-918A-457F-BC9E-F2AC2EFC1F86@anarazel.de>
 <20200625225853.GA11137@alvherre.pgsql>
 <20200708140037.GI3125@tamriel.snowman.net>
 <99b047386525b87df1e611335c2e57a6cdef346b.camel@j-davis.com>
 <20200709140330.GO3125@tamriel.snowman.net>
 <CAH2-WzmB04dxgRgjDGvDbugEFfHxMi2pyuJqsYr30x1RAEqSXg@mail.gmail.com>
 <20200709225840.GE12375@tamriel.snowman.net>
 <20200710001852.GC26220@telsasoft.com>
 <6963442a2654bd56d386d5a253586815d0c3a17f.camel@j-davis.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="Xssso5lpTBgMxDfe"
Content-Disposition: inline
In-Reply-To: <6963442a2654bd56d386d5a253586815d0c3a17f.camel@j-davis.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--Xssso5lpTBgMxDfe
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Jeff Davis (pgsql@j-davis.com) wrote:
> In principle, Stephen is right: the v12 behavior is a bug, lots of
> people are unhappy about it, it causes real problems, and it would not
> be acceptable if proposed today. Otherwise I wouldn't have spent the
> time to fix it.=20
>=20
> Similarly, potential regressions are not the "fault" of my feature --
> they are the fault of the limitations of work_mem, the limitations of
> the planner, the wrong expectations from customers, or just
> happenstance.=20

Exactly.

> But at a certain point, I have to weigh the potential anger of
> customers hitting regressions versus the potential anger of hackers
> seeing a couple extra GUCs. I have to say that I am more worried about
> the former.

We work, quite intentionally, to avoid having a billion knobs that
people have to understand and to tune.  Yes, we could create a bunch of
new GUCs to change all kinds of behavior, and we could add hints while
we're at it, but there's been quite understandable and good pressure
against doing so because much of the point of this database system is
that it should be figuring out the best plan on its own and within the
constraints that users have configured.

> If there is some more serious consequence of adding a GUC that I missed
> in this thread, please let me know. Otherwise, I intend to commit a new
> GUC shortly that will enable users to bypass work_mem for HashAgg, just
> as in v12.

I don't think this thread has properly considered that every new GUC,
every additional knob that we create, increases the complexity of the
system for users to have to deal with and, in some sense, creates a
failure of ours to be able to just figure out what the right answer
is.  For such a small set of users, who somehow have a problem with a
Sort taking up more memory but are fine with HashAgg doing so, I don't
think the requirement is met that this is a large enough issue to
warrant a new GUC.  Users who are actually hit by this in a negative way
have an option- increase work_mem to reflect what was actually happening
already.  I seriously doubt that we'd get tons of users complaining
about that answer or asking us to have something separate from that, and
we'd avoid adding some new GUC that has to be explained to every new
user to the system and complicate the documentation that explains how
work_mem works.

Thanks,

Stephen

--Xssso5lpTBgMxDfe
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJfCHxnAAoJEO1sijiDR2RVxjcQAJhNCD+Si5+HWXqnG3qbFfJB
ZCgdHqsKS07Mmk43piblq5zZoFgUCHXhoBbSKFHBFrVLM+fdJ8Q+i5OXqazy8N9p
wUMd5T75ySx7jyxZU4vR0dKJFjzVyQwjB6yUufllQC/qj1HLDuFAe/fxXdti7P73
hxQhu79VSHXGQ8sm70LuOLBIpAKcoyvBqQtxFjk19hJ/3ClpGFs3b/ISxAOXlEor
zYK6cFR3ddQB0N6vufqwkukUhs+GIXSaeA2GNQICohntTyAss2jU4lBvUoSuQbUI
i/utiMyac31VxC1gxR3zgeuDoW00KPqbHNQP3guDnrl+yduzy9BJPs3uPAO0J3Hs
Dhg4GfvM/FqCGFuGpLnGyF1Wiz8g6mv9SuwtjFh/Fso06vEFZI7FDZWqyzYXkzgP
I+OiEiOzP9DZqcbkGNlXw5twRkDS1AIemLyfOL6XuBGtM3rmD2EUuunZUss80rtK
AUSn7cgkPKIXqXUTHhO3SS10dxsB8XxcxP5QoCvP5NXFwGu4GhfMSDgqh7A0ECMb
Fd/aceLW7lI4TV/ziEh5RFIwuovCYrVdpBulYfCrZHwkc+4kGO6bC3TMVQ85zfxG
rLTCr/aYkvBVd+dg1BtNErL3PTJ6B4IT3V3I1XRX+T/HPF9+ukZekQlXBi62mXeX
lUPqW7tcqAvyxwgkiakW
=T/E1
-----END PGP SIGNATURE-----

--Xssso5lpTBgMxDfe--