Date: Tue, 3 Dec 2019 14:07:04 -0300
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andres Freund <andres@anarazel.de>,
	Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	James Coleman <jtc331@gmail.com>,
	PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #16104: Invalid DSA Memory Alloc Request in Parallel Hash
Message-ID: <20191203170704.GA28076@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CA+hUKGJVgKknBG69fRyj_NRTjVk5mJqcZ7--uNS2C373Ry7KRg@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Precedence: bulk

Hello

I can make no useful comment on the correctness of the new bit
distribution.  But I can make two comments on this part:

On 2019-Nov-21, Thomas Munro wrote:

> So here's a draft steal-the-bucket-bits patch.  Yeah,
> reverse_bits_u32() may be in the wrong header.

I think pg_bitutils.h is the right place in master, but that file didn't
exist earlier.  I think access/hash.h is the right place in older
branches, which is where hash_any() used to be.

> But is it too slow?
> On my desktop I can call ExecGetBucketAndBatch() 353 million times per
> second (~2.8ns), and unpatched gets me 656 million/sec (~1.5ns)
> (though that includes a function call, and when Hash does it it's
> inlined), but it's only slower when nbatch > 1 due to the condition.

If the whole query takes hours (or even minutes) to run, then adding one
second of runtime is not going to change things in any noticeable way.
I'd rather have the extreme cases working and take additional 1.3ns per
output row, than not work at all.

> To put that number into persepective, I can hash 10 million single-int
> tuples from a prewarmed seq scan in 2.5s without batching or
> parallelism, so that's 250ns per tuple.  This'd be +0.4% of that, and
> I do see it in a few more samples with my profiler, but it's still
> basically nothing, and lost in the noise with other noisy partitioning
> overheads like IO.  Thoughts?

That summarizes it well for me: yes, it's a slowdown, yes it's barely
measurable.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services