From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Wrong hash table size calculation in Parallel Hash Join
Date: Sat, 18 Jan 2020 13:40:48 +0500
Message-ID: <5fc5ac28-8ed3-c2d7-8f51-761b2761808c@postgrespro.ru> (raw)
While running the Join Order Benchmark [1] test on my laptop with 8GB
RAM I saw strange situation where linux (Ubuntu 18, PostgreSQL master
branch) killed a backend (or a parallel worker in some cases) with signal 9.
Dmesg showed an error:
[176313.266426] Out of memory: Kill process 16011 (postgres) score 135
or sacrifice child
[176313.266432] Killed process 16011 (postgres) total-vm:1602320kB,
anon-rss:1325908kB, file-rss:1136kB, shmem-rss:44612kB
I spend time investigating and saw the problem.
Calculation of hash table size based on GUC "work_mem". But if we have
huge relation, we divides it into the batches, stored into a shared
tuplestores. At the first write into the batch, tuplestore allocates
memory buffer:
accessor->write_chunk = (SharedTuplestoreChunk *)
MemoryContextAllocZero(accessor->context, STS_CHUNK_PAGES * BLCKSZ);
if we have many batches, we have many additional memory allocations. In
my case work_mem=64MB, nbatches=65000 and we need about 2GB of
additional memory.
I don't specialized in the Parallel Hash Join code before, but
considering the code, more accurate calculation of the size of hash
table will be based on the quadratic equation, something like this:
M_h = M+sqrt(M^2 -M_b*RelSize)/2
here M - work_mem GUC; M_h - estimation of the hash table size; RelSize
- size of relation in bytes; M_b=4*STS_CHUNK_PAGES * BLCKSZ.
In the case of M^2 < 4*STS_CHUNK_PAGES * BLCKSZ we need to increase
work_mem locally.
As some sketch of the solution i prepared the patch (see in attachment).
If this is a significant problem, I'm ready to continue the solution.
[1] https://github.com/gregrahn/join-order-benchmark
--
Andrey Lepikhov
Postgres Professional
https://postgrespro.com
The Russian Postgres Company
Attachments:hash_table_size_calculation.patch (text/x-patch, 813B) downloaddiff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.cindex 568938667f..a5f6adfbb1 100644--- a/src/backend/executor/nodeHash.c+++ b/src/backend/executor/nodeHash.c@@ -700,8 +700,13 @@ ExecChooseHashTableSize(double ntuples, int tupwidth, bool useskew,
/*
* Target in-memory hashtable size is work_mem kilobytes.
*/
- hash_table_bytes = work_mem * 1024L;-+ if (work_mem <= sqrt(4. * 4 * BLCKSZ * inner_rel_bytes) / 1024)+ hash_table_bytes = (long) sqrt(4. * 4 * BLCKSZ * inner_rel_bytes) / 2;+ else+ {+ long wm = work_mem * 1024L;+ hash_table_bytes = (wm + sqrt(wm*wm - 4. * 4 * BLCKSZ * inner_rel_bytes)) / 2;+ }
/*
* Parallel Hash tries to use the combined work_mem of all workers to
* avoid the need to batch. If that won't work, it falls back to work_mem
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: a.lepikhov@postgrespro.ru
Subject: Re: Wrong hash table size calculation in Parallel Hash Join
In-Reply-To: <5fc5ac28-8ed3-c2d7-8f51-761b2761808c@postgrespro.ru>
References: <5fc5ac28-8ed3-c2d7-8f51-761b2761808c@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox