pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: pgsql-committers <pgsql-committers@postgresql.org>
Subject: Re: pgsql: Add parallel-aware hash joins.
Date: Wed, 03 Jan 2018 13:03:39 -0500
Message-ID: <18804.1515002619@sss.pgh.pa.us> (raw)
In-Reply-To: <CAEepm=34PDuR69kfYVhmZPgMdy8pSA-MYbpesEN1SR+2oj3Y+w@mail.gmail.com>
References: <E1eRwXy-0004IK-TO@gemulon.postgresql.org>
	<20171221092940.gbxcszbzsr5khsfr@alap3.anarazel.de>
	<20171221095550.d3e7xhoakcowhu5b@alap3.anarazel.de>
	<CAEepm=3eS6ra8oBCYsYYjPNsPqb2R_TaGQK8SGRFq_7gbP8RUg@mail.gmail.com>
	<CAEepm=0WxwzpHVHt3PcWHBV=L3k3FDb6dvMq1A2Li49LGBa7TA@mail.gmail.com>
	<20171222082203.fl6rpzlqnm4h7h6h@alap3.anarazel.de>
	<CAEepm=04WhyY65BpE2g_8E==ayhBXjV9JbyR-+9N4McmY3SQkQ@mail.gmail.com>
	<30219.1514428346@sss.pgh.pa.us>
	<CAEepm=1OyZcCAxAEWYhmeg0qmWyYF2O2yW7s7A4uVTt-dgRRyQ@mail.gmail.com>
	<CAEepm=2H80cY=DncWKjoSiwaX=xbLcW7dqee+1H5-CQ6pJJnAQ@mail.gmail.com>
	<CAEepm=3GL7H3vxQb0LTJ9KnKhpnvaqOK=yn9p-O8zyjxRgXntA@mail.gmail.com>
	<30655.1514673257@sss.pgh.pa.us>
	<CAEepm=3SeFvsfnnOLSA3tLtBe-rtyL=c+vfzyPCsViBjk521qw@mail.gmail.com>
	<4001.1514678419@sss.pgh.pa.us>
	<CAEepm=2WFQrwDD4rEVKjdPBQ_di_0n=xYST1MDRsucZ=JgDPxg@mail.gmail.com>
	<18190.1514943510@sss.pgh.pa.us>
	<CAEepm=34PDuR69kfYVhmZPgMdy8pSA-MYbpesEN1SR+2oj3Y+w@mail.gmail.com>

Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Wed, Jan 3, 2018 at 2:38 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Hm.  That could do it, except it doesn't really account for the observed
>> result that slower single-processor machines seem more prone to the
>> bug.  Surely they should be less likely to get multiple workers activated.

> It plans for 8 batches, and then usually but not always goes to 16 at
> execution time depending on timing.  It doesn't happen for me with
> 128kB (the setting used in the regression test), but I think the
> affected BF machines are all 32 bit systems that have MAXIMUM_ALIGNOF
> 8 and therefore use a bit more space, whereas my machines have
> MAXIMUM_ALIGNOF 4 in a 32 bit build, so that would explain the
> different location of this unstable border.

Ah, that might do it.  You're right that gaur/pademelon are in that class.

> We could probably fix the
> failures by simply increasing work_mem out of that zone, but I'm
> hoping to fix the problem in a more principled way.  More soon.

Meh.  I think we're going to end up having to pick a modified test
case that's further away from the chunk size threshold.  I do not
think it is possible to predict this runtime behavior exactly at
plan time, nor am I convinced that expending planner cycles on a
somewhat closer estimate is a useful use of planning time.

>> I'm also wondering why the non-parallel path seems to prefer to allocate
>> in units of HASH_CHUNK_SIZE + HASH_CHUNK_HEADER_SIZE while the parallel
>> path targets allocations of exactly HASH_CHUNK_SIZE,

> That is intentional: dsa.c sucks at allocating 32KB + a tiny bit
> because it works in 4KB pages for large allocations, so I wanted to
> make HASH_CHUNK_SIZE the total size that arrives into dsa_allocate().
> The non-parallel path has similar problems on some libc
> implementations, as we discussed over here:
> https://www.postgresql.org/message-id/flat/29770.1511495642%40sss.pgh.pa.us

Oh, right, I'd forgotten about that discussion.  I think it would be
good to adjust hashjoin so that both paths are targeting 32KB total;
but you are right that there's not a lot of point in fooling with the
non-parallel path until we add some way of accounting for aset.c's
overhead too.  We can leave that for later.

>> BTW, I'm seeing a few things that look bug-like about
>> ExecParallelHashTuplePrealloc, ...

> Right.  Fixed in the attached.

Pushed, plus an additional Assert to clarify that we're expecting
ExecParallelHashTuplePrealloc's caller to have already maxalign'ed
the request size.

			regards, tom lane




view thread (66+ messages)  latest in thread

Message-ID: <18804.1515002619@sss.pgh.pa.us>
Permalink:  ../18804.1515002619@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/18804.1515002619@sss.pgh.pa.us

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us, thomas.munro@enterprisedb.com, andres@anarazel.de
  Subject: Re: pgsql: Add parallel-aware hash joins.
  In-Reply-To: <18804.1515002619@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox