From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>
Cc: hackers <pgsql-hackers@postgresql.org>
Subject: Re: accounting for memory used for BufFile during hash joins
Date: Tue, 7 May 2019 15:17:42 +0200
Message-ID: <20190507131742.nixczzyyyuuoig7m@development> (raw)
In-Reply-To: <27902.1557199108@sss.pgh.pa.us>
References: <20190504003414.bulcbnge3rhwhcsh@development>
<CAAKRu_ZTy=a2jqj0++jDwke1_ssJTBB0ezU2t8OwogmXDbYdYw@mail.gmail.com>
<CA+hUKG+ppih9DtZwGJh4KMZXdtD=c9+fYoS1CSeBQBs9JzZtbA@mail.gmail.com>
<20190507031511.lvp6bqn2e5dpsnga@development>
<27902.1557199108@sss.pgh.pa.us>
On Mon, May 06, 2019 at 11:18:28PM -0400, Tom Lane wrote:
>Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
>> Do we actually check how many duplicates are there during planning?
>
>Certainly that's part of the planner's cost estimates ... but it's
>only as good as the planner's statistical knowledge.
>
I'm looking at the code, and the only place where I see code dealing with
MCVs (probably the best place for info about duplicate values) is
estimate_hash_bucketsize in final_cost_hashjoin. That's not quite what I
had in mind - I was thinking more about something along the lines "See the
larget group of duplicate values, disable hash join if it can't fit into
work_mem at all."
Of course, if the input estimates are off, that may not work too well. It
would certainly not help the query failing with OOM, because that was a
case of severe underestimate.
Or did you mean some other piece of code that I have missed.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, tgl@sss.pgh.pa.us, thomas.munro@gmail.com, melanieplageman@gmail.com
Subject: Re: accounting for memory used for BufFile during hash joins
In-Reply-To: <20190507131742.nixczzyyyuuoig7m@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox