Date: Thu, 25 Jun 2020 15:48:24 -0700
User-Agent: K-9 Mail for Android
In-Reply-To: <20200625224422.GA9653@alvherre.pgsql>
References: <20200625224422.GA9653@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: Default setting for enable_hashagg_disk
To: Alvaro Herrera <alvherre@2ndquadrant.com>
CC: Bruce Momjian <bruce@momjian.us>,Jeff Davis <pgsql@j-davis.com>,Robert
 Haas <robertmhaas@gmail.com>,David Rowley <dgrowleyml@gmail.com>,Justin
 Pryzby <pryzby@telsasoft.com>,Melanie Plageman
 <melanieplageman@gmail.com>,Tomas Vondra
 <tomas.vondra@2ndquadrant.com>,"pgsql-hackers@postgresql.org"
 <pgsql-hackers@postgresql.org>
From: Andres Freund <andres@anarazel.de>
Message-ID: <85F9F4F0-918A-457F-BC9E-F2AC2EFC1F86@anarazel.de>
Precedence: bulk

Hi,=20

On June 25, 2020 3:44:22 PM PDT, Alvaro Herrera <alvherre@2ndquadrant=2Eco=
m> wrote:
>On 2020-Jun-25, Andres Freund wrote:
>
>> > What are people doing for those cases already?  Do we have an
>> > real-world queries that are a problem in PG 13 for this?
>>=20
>> I don't know about real world, but it's pretty easy to come up with
>> examples=2E
>>=20
>> query:
>> SELECT a, array_agg(b) FROM (SELECT generate_series(1, 10000)) a(a),
>(SELECT generate_series(1, 10000)) b(b) GROUP BY a HAVING
>array_length(array_agg(b), 1) =3D 0;
>>=20
>> work_mem =3D 4MB
>>=20
>> 12      18470=2E012 ms
>> HEAD    44635=2E210 ms
>>=20
>> HEAD causes ~2=2E8GB of file IO, 12 doesn't cause any=2E If you're IO
>> bandwidth constrained, this could be quite bad=2E
>
>=2E=2E=2E however, you can pretty much get the previous performance back =
by
>increasing work_mem=2E  I just tried your example here, and I get 32
>seconds of runtime for work_mem 4MB, and 13=2E5 seconds for work_mem 1GB
>(this one spills about 800 MB); if I increase that again to 1=2E7GB I get
>no spilling and 9 seconds of runtime=2E  (For comparison, 12 takes 15=2E7
>seconds regardless of work_mem)=2E
>
>My point here is that maybe we don't need to offer a GUC to explicitly
>turn spilling off; it seems sufficient to let users change work_mem so
>that spilling will naturally not occur=2E  Why do we need more?

That's not really a useful escape hatch, because I'll often lead to other =
nodes using more memory=2E

Andres
--=20
Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E