From: Tom Lane <tgl@sss.pgh.pa.us>
To: Hartranft, Robert M. (GSFC-423.0)[RAYTHEON CO] <robert.m.hartranft@nasa.gov>
Cc: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Subject: Re: question on hash joins
Date: Thu, 19 Oct 2017 12:15:13 -0400
Message-ID: <24789.1508429713@sss.pgh.pa.us> (raw)
In-Reply-To: <E46ADDFD-C631-441C-9859-7A956CFBCACE@nasa.gov>
References: <60286BB5-0A38-42C8-A673-5289CA6D2354@nasa.gov>
<63F14C4C-1518-4EAA-9EBE-87222988F073@nasa.gov>
<22675.1508361426@sss.pgh.pa.us>
<B4DE522C-7B7B-47DC-9143-B3087A7487D5@nasa.gov>
<30523.1508422449@sss.pgh.pa.us>
<E46ADDFD-C631-441C-9859-7A956CFBCACE@nasa.gov>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-admin>
"Hartranft, Robert M. (GSFC-423.0)[RAYTHEON CO]" <robert.m.hartranft@nasa.gov> writes:
> Given that the hash would only contain keys and values needed for supporting
> the query I am having a hard time understanding why I am exceeding the
> 10 GB temp_file_limit.
Because *both sides* of the join are getting dumped to temp files.
This is necessary when the hash requires multiple batches. All
but the first batch of hash keys get written out to files, and then
we reload each batch of the inner relation into the in-memory table
and scan the corresponding batch file from the outer relation.
If you can make work_mem large enough to hold the inner relation
then the problem should go away. Note though that the per-row
overhead is significantly larger in the in-memory representation;
don't have an estimate for that offhand.
regards, tom lane
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: tgl@sss.pgh.pa.us, robert.m.hartranft@nasa.gov
Subject: Re: question on hash joins
In-Reply-To: <24789.1508429713@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox