Message-ID: <7a1f5e5b-632f-4cc5-8078-b18a848e8924@postgrespro.ru>
Date: Thu, 29 Feb 2024 10:36:55 +0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: BUG #18349: ERROR: invalid DSA memory alloc request size
 1811939328, CONTEXT: parallel worker
Content-Language: en-US
To: Tomas Vondra <tomas.vondra@enterprisedb.com>,
 Alexey Ermakov <alexey.ermakov@dataegret.com>,
 pgsql-bugs@lists.postgresql.org
References: <18349-83d33dd3d0c855c3@postgresql.org>
 <512ef568-336e-4087-ba23-f22b006d329d@postgrespro.ru>
 <df93756b-e011-1b09-7bbb-18e6e1ec3ae2@dataegret.com>
 <df297e10-d50f-497d-b278-86a6bce1c44e@enterprisedb.com>
From: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
Organization: Postgres Professional
In-Reply-To: <df297e10-d50f-497d-b278-86a6bce1c44e@enterprisedb.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/7a1f5e5b-632f-4cc5-8078-b18a848e8924%40postgrespro.ru>
Precedence: bulk

On 21/2/2024 19:52, Tomas Vondra wrote:
> Hi,
> 
> On 2/20/24 15:05, Alexey Ermakov wrote:
>> On 2024-02-16 20:40, Andrei Lepikhov wrote:
>>> Interesting. It correlates with one performance issue I have been
>>> trying to catch already 3 months. Could you provide some reproduction
>>> of that behavior?
>>>
>> Yes, I'm still trying to make reproducer, it will take some time. Thanks.
>>
> It's a bit weird it needs 1.8GB of memory, but perhaps that's also
> linked to the number of batches, somehow?
I found one possible weak point in the code of PHJ:
ExecParallelHashJoinSetUpBatches:

pstate->batches = dsa_allocate0(hashtable->area,
	EstimateParallelHashJoinBatch(hashtable) * nbatch);

It could explain why we have such a huge memory allocation with a size 
not bonded to a power of 2.
Also, it makes more sense that we already have a second report [1] with 
the same value of DSA allocation.
So, can Alexey apply a trivial patch and check whether the bug has gone?

[1] 
https://www.postgresql.org/message-id/flat/PAYP264MB3368AF4D63704E370B204C39FF582%40PAYP264MB3368.FRAP264.PROD.OUTLOOK.COM

-- 
regards,
Andrei Lepikhov
Postgres Professional