From: Tom Lane <tgl@sss.pgh.pa.us>
To: David Rowley <dgrowleyml@gmail.com>
cc: "ldh@laurent-hasson.com" <ldh@laurent-hasson.com>,
        Peter Geoghegan <pg@bowt.ie>, Justin Pryzby <pryzby@telsasoft.com>,
        "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Re: Big performance slowdown from 11.2 to 13.3
In-reply-to: 
 <CAApHDvo+Gb80BVPKYCwHbLa7a7ZXp9ziq9URrpQaKdvXsd46bg@mail.gmail.com>
References: 
 <MN2PR15MB25601E80A9B6D1BA6F592B1985E39@MN2PR15MB2560.namprd15.prod.outlook.com>
 <20210721191528.GC19620@telsasoft.com>
 <MN2PR15MB256016E33F00DE60B96F02D385E39@MN2PR15MB2560.namprd15.prod.outlook.com>
 <CAH2-Wzm3rSAWHyENb3T0X8M_qdYjF1t7rntkMJ2kDiuhVOHx3g@mail.gmail.com>
 <MN2PR15MB2560C1A890DC07F9359FB56285E39@MN2PR15MB2560.namprd15.prod.outlook.com>
 <732758.1626911007@sss.pgh.pa.us>
 <MN2PR15MB2560778595BB7AF6E691B26785E39@MN2PR15MB2560.namprd15.prod.outlook.com>
 <MN2PR15MB2560D429866BD65A70574A6C85E49@MN2PR15MB2560.namprd15.prod.outlook.com>
 <CAApHDvqoF7y7SQjGT=940njnAd_CUcUG6oRYChcP_+eJyuNiug@mail.gmail.com>
 <MN2PR15MB25603D3CE06C8F9C312EA58985E49@MN2PR15MB2560.namprd15.prod.outlook.com>
 <MN2PR15MB2560E4574A6E2B5AED2C03CB85E49@MN2PR15MB2560.namprd15.prod.outlook.com>
 <784576.1626968702@sss.pgh.pa.us> <785218.1626969409@sss.pgh.pa.us>
 <CAApHDvo+Gb80BVPKYCwHbLa7a7ZXp9ziq9URrpQaKdvXsd46bg@mail.gmail.com>
Comments: In-reply-to David Rowley <dgrowleyml@gmail.com>
	message dated "Fri, 23 Jul 2021 04:04:00 +1200"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <786599.1626970898.1@sss.pgh.pa.us>
Date: Thu, 22 Jul 2021 12:21:38 -0400
Message-ID: <786600.1626970898@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/786600.1626970898%40sss.pgh.pa.us>
Precedence: bulk

David Rowley <dgrowleyml@gmail.com> writes:
> On Fri, 23 Jul 2021 at 03:56, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> So basically, we now have a hard restriction that hashaggs can't use
>> more than INT_MAX kilobytes, or approximately 2.5GB, and this use case
>> is getting eaten alive by that restriction.  Seems like we need to
>> do something about that.

> Hmm, math check?

Yeah, I should have said "2GB plus palloc slop".  It doesn't surprise
me a bit that we seem to be eating another 20% on top of the nominal
limit.

I think the right fix here is to remove the cap, which will require
changing get_hash_mem to return double, and then maybe some cascading
changes --- I've not looked at its callers.

			regards, tom lane