From: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
To: Alexey Ermakov <alexey.ermakov@dataegret.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker
Date: Tue, 12 Mar 2024 13:44:01 +0700
Message-ID: <7d763a6d-fad7-49b6-beb0-86f99ce4a6eb@postgrespro.ru> (raw)
In-Reply-To: <a2f83556-5ef0-ddca-c63d-bbcbf5054335@dataegret.com>
References: <18349-83d33dd3d0c855c3@postgresql.org>
<512ef568-336e-4087-ba23-f22b006d329d@postgrespro.ru>
<df93756b-e011-1b09-7bbb-18e6e1ec3ae2@dataegret.com>
<df297e10-d50f-497d-b278-86a6bce1c44e@enterprisedb.com>
<7a1f5e5b-632f-4cc5-8078-b18a848e8924@postgrespro.ru>
<2ff6260b-8722-b301-cf7e-4e66a159778c@dataegret.com>
<90092199-55cd-4fa3-a608-9a41d8e76c44@postgrespro.ru>
<a2f83556-5ef0-ddca-c63d-bbcbf5054335@dataegret.com>
On 11/3/2024 15:12, Alexey Ermakov wrote:
> Would you like to get any additional info from gdb (like number of
> batches) ?
Can you try to reproduce the case with simple patch attached?
--
regards,
Andrei Lepikhov
Postgres Professional
Attachments:0001-Strictly-limit-max-size-of-batches-in-Parallel-Hash-.patch (text/plain, 1.8K) download
From 02d74361c28a930135549ad679b7baa061d6a90b Mon Sep 17 00:00:00 2001
From: "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>
Date: Tue, 12 Mar 2024 13:39:29 +0700
Subject: [PATCH] Strictly limit max size of batches in Parallel Hash Join by
DSA limits.
Because batch descriptors allocates in the DSA area we must limit their number
by the limitations of maximum size of DSA area that can be allocated.
---
src/backend/executor/nodeHash.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.cindex 61480733a1..d95700a4f7 100644--- a/src/backend/executor/nodeHash.c+++ b/src/backend/executor/nodeHash.c@@ -1145,6 +1145,7 @@ ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable)
/* Allocate new larger generation of batches. */
Assert(hashtable->nbatch == pstate->nbatch);
+
ExecParallelHashJoinSetUpBatches(hashtable, new_nbatch);
Assert(hashtable->nbatch == pstate->nbatch);
@@ -1246,6 +1247,8 @@ ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable)
{
bool space_exhausted = false;
bool extreme_skew_detected = false;
+ int max_batches = (int) pg_prevpower2_32((uint32)+ (MaxAllocSize / EstimateParallelHashJoinBatch(hashtable)));
/* Make sure that we have the current dimensions and buckets. */
ExecParallelHashEnsureBatchAccessors(hashtable);
@@ -1276,7 +1279,7 @@ ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable)
}
/* Don't keep growing if it's not helping or we'd overflow. */
- if (extreme_skew_detected || hashtable->nbatch >= INT_MAX / 2)+ if (extreme_skew_detected || hashtable->nbatch >= max_batches)
pstate->growth = PHJ_GROWTH_DISABLED;
else if (space_exhausted)
pstate->growth = PHJ_GROWTH_NEED_MORE_BATCHES;
--
2.44.0
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: a.lepikhov@postgrespro.ru, alexey.ermakov@dataegret.com, tomas.vondra@enterprisedb.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker
In-Reply-To: <7d763a6d-fad7-49b6-beb0-86f99ce4a6eb@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox