Date: Sun, 10 Nov 2019 23:02:59 +0100
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@gmail.com>, James Coleman <jtc331@gmail.com>,
	PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #16104: Invalid DSA Memory Alloc Request in Parallel Hash
Message-ID: <20191110220259.r4x6zon4zjjxz2vj@development>
References: <20191109182735.ewficcv2hibbyzl4@development>
 <CAAaqYe8=xUUwYp+XO6Y3YNqL0Nk+UXrqxYjT46Z+q6b-5U1N1Q@mail.gmail.com>
 <CA+hUKGJFNg0XqXiBNzOeTPsCa4tb-a00Sv5EA7jD+w26ATyUyA@mail.gmail.com>
 <CAAaqYe-VHx2O1k1x-wML3_1GHCPajugvMify=a_4cq-aHb=ZVg@mail.gmail.com>
 <CA+hUKGL0T7-Lsbp-dda5JWubAXEHMkL7j94-XW9ZhXDb4sn2+A@mail.gmail.com>
 <CAAaqYe9sMXm8e2+9n6KK2OTMdH3wBku06cpTcchu7k2wX2dAqA@mail.gmail.com>
 <20191110172925.mlkglpwzwego3wtp@development>
 <CAAaqYe_w=Q2F9tM0EyUvLn=CrwM99S3w9LE=MgghxKa6SRwdJw@mail.gmail.com>
 <CA+hUKGLyafKXBMFqZCSeYikPbdYURbwr+jP6TAy8sY-8LO0V+Q@mail.gmail.com>
 <8966.1573421649@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: <8966.1573421649@sss.pgh.pa.us>
Precedence: bulk

On Sun, Nov 10, 2019 at 04:34:09PM -0500, Tom Lane wrote:
>Thomas Munro <thomas.munro@gmail.com> writes:
>> I think I see what's happening: we're running out of hash bits.
>
>Ugh.
>
>> Besides switching to 64 bit hashes so that we don't run out of
>> information (clearly a good idea), what other options do we have?
>
>If the input column is less than 64 bits wide, this is an illusory
>solution, because there can't really be that many independent hash bits.
>I think we need to have some constraint that keeps us from widening
>the batch+bucket counts to more hash bits than we have.
>

That's not quite correct - it's a solution/improvement as long as the
values have more than 32 bits of information. I.e. it works even for
values that are less than 64 bits wide, as long as there's more than 32
bits of information (assuming the hash does not lost the extra bits).

In the case discussed in this text we're dealing with a citext column,
with average length ~8 chars.  So it probably has less than 64 bits of
information, because it's likely just alphanumeric, or something like
that. But likely more than 32 bits.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services