Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e5DVR-0007RW-Q0 for pgsql-admin@arkaria.postgresql.org; Thu, 19 Oct 2017 16:17:13 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1e5DVQ-0000Ee-Ay for pgsql-admin@arkaria.postgresql.org; Thu, 19 Oct 2017 16:17:12 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1e5DTc-0005PH-7h for pgsql-admin@postgresql.org; Thu, 19 Oct 2017 16:15:20 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1e5DTY-0005IO-L7 for pgsql-admin@postgresql.org; Thu, 19 Oct 2017 16:15:19 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.14.4/8.14.4) with ESMTP id v9JGFDNT024790; Thu, 19 Oct 2017 12:15:13 -0400 From: Tom Lane To: "Hartranft, Robert M. (GSFC-423.0)[RAYTHEON CO]" cc: "pgsql-admin@postgresql.org" Subject: Re: question on hash joins In-reply-to: References: <60286BB5-0A38-42C8-A673-5289CA6D2354@nasa.gov> <63F14C4C-1518-4EAA-9EBE-87222988F073@nasa.gov> <22675.1508361426@sss.pgh.pa.us> <30523.1508422449@sss.pgh.pa.us> Comments: In-reply-to "Hartranft, Robert M. (GSFC-423.0)[RAYTHEON CO]" message dated "Thu, 19 Oct 2017 15:48:05 -0000" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <24788.1508429713.1@sss.pgh.pa.us> Content-Transfer-Encoding: quoted-printable Date: Thu, 19 Oct 2017 12:15:13 -0400 Message-ID: <24789.1508429713@sss.pgh.pa.us> List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-admin Precedence: bulk Sender: pgsql-admin-owner@postgresql.org "Hartranft, Robert M. (GSFC-423.0)[RAYTHEON CO]" writes: > Given that the hash would only contain keys and values needed for support= ing > the query I am having a hard time understanding why I am exceeding the=20 > 10 GB temp_file_limit. Because *both sides* of the join are getting dumped to temp files. This is necessary when the hash requires multiple batches. All but the first batch of hash keys get written out to files, and then we reload each batch of the inner relation into the in-memory table and scan the corresponding batch file from the outer relation. If you can make work_mem large enough to hold the inner relation then the problem should go away. Note though that the per-row overhead is significantly larger in the in-memory representation; don't have an estimate for that offhand. regards, tom lane --=20 Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin