pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Justin Pryzby <pryzby@telsasoft.com>
To: Melanie Plageman <melanieplageman@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Richard Guo <guofenglinux@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Wrong results from Parallel Hash Full Join
Date: Thu, 20 Apr 2023 10:50:45 -0500
Message-ID: <ZEFfVUeNzQaeHy7R@telsasoft.com> (raw)
In-Reply-To: <CAAKRu_YFUC+Sh9969RJ7GCSX=nPOQLVJp9UHK_iHtVT=wkBEKQ@mail.gmail.com>
References: <20230412181452.cyfduw2ka2atqmii@awork3.anarazel.de>
	<CAAKRu_Z=rzXjxVtA7gPe2bmBk7F6e4_pe8iT5DmnYFOgWPxMGQ@mail.gmail.com>
	<CAAKRu_Ybw_0MDNTW_jg3gndXs7F6H8MUZkbY2iMtSHeS5L97hw@mail.gmail.com>
	<CA+hUKGKuLORE0OQ20sFrfdp=5tKVvamB-FnnCS03UZE+o8UFjQ@mail.gmail.com>
	<CAAKRu_av+NCVPXkLLTBG2VZcAk83FrzR9z-kZ-NQ+1kai9Tibw@mail.gmail.com>
	<20230419151704.xkkobswffutslj34@liskov>
	<ZEAh6CewmARbDVuN@telsasoft.com>
	<20230419192051.xkgyj76akczpbjbs@awork3.anarazel.de>
	<ZECKP5NwoDGT3+RN@telsasoft.com>
	<CAAKRu_YFUC+Sh9969RJ7GCSX=nPOQLVJp9UHK_iHtVT=wkBEKQ@mail.gmail.com>

On Wed, Apr 19, 2023 at 08:47:07PM -0400, Melanie Plageman wrote:
> On Wed, Apr 19, 2023 at 8:41 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> >
> > On Wed, Apr 19, 2023 at 12:20:51PM -0700, Andres Freund wrote:
> > > On 2023-04-19 12:16:24 -0500, Justin Pryzby wrote:
> > > > On Wed, Apr 19, 2023 at 11:17:04AM -0400, Melanie Plageman wrote:
> > > > > Ultimately this is probably fine. If we wanted to modify one of the
> > > > > existing tests to cover the multi-batch case, changing the select
> > > > > count(*) to a select * would do the trick. I imagine we wouldn't want to
> > > > > do this because of the excessive output this would produce. I wondered
> > > > > if there was a pattern in the tests for getting around this.
> > > >
> > > > You could use explain (ANALYZE).  But the output is machine-dependant in
> > > > various ways (which is why the tests use "explain analyze so rarely).
> > >
> > > I think with sufficient options it's not machine specific.
> >
> > It *can* be machine specific depending on the node type..
> >
> > In particular, for parallel workers, it shows "Workers Launched: ..",
> > which can vary even across executions on the same machine.  And don't
> > forget about "loops=".
> >
> > Plus:
> > src/backend/commands/explain.c: "Buckets: %d  Batches: %d  Memory Usage: %ldkB\n",
> >
> > > We have a bunch of
> > >  EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF) ..
> > > in our tests.
> >
> > There's 81 uses of "timing off", out of a total of ~1600 explains.  Most
> > of them are in partition_prune.sql.  explain analyze is barely used.
> >
> > I sent a patch to elide the machine-specific parts, which would make it
> > easier to use.  But there was no interest.
> 
> While I don't know about other use cases, I would have used that here.
> Do you still have that patch laying around? I'd be interested to at
> least review it.

https://commitfest.postgresql.org/41/3409/





view thread (22+ messages)  latest in thread

Message-ID: <ZEFfVUeNzQaeHy7R@telsasoft.com>
Permalink:  ../ZEFfVUeNzQaeHy7R@telsasoft.com/
Also on:    postgresql.org/message-id/ZEFfVUeNzQaeHy7R@telsasoft.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: pryzby@telsasoft.com, melanieplageman@gmail.com, andres@anarazel.de, thomas.munro@gmail.com, guofenglinux@gmail.com
  Subject: Re: Wrong results from Parallel Hash Full Join
  In-Reply-To: <ZEFfVUeNzQaeHy7R@telsasoft.com>
  References: <20230412181452.cyfduw2ka2atqmii@awork3.anarazel.de> <CAAKRu_Z=rzXjxVtA7gPe2bmBk7F6e4_pe8iT5DmnYFOgWPxMGQ@mail.gmail.com> <CAAKRu_Ybw_0MDNTW_jg3gndXs7F6H8MUZkbY2iMtSHeS5L97hw@mail.gmail.com> <CA+hUKGKuLORE0OQ20sFrfdp=5tKVvamB-FnnCS03UZE+o8UFjQ@mail.gmail.com> <CAAKRu_av+NCVPXkLLTBG2VZcAk83FrzR9z-kZ-NQ+1kai9Tibw@mail.gmail.com> <20230419151704.xkkobswffutslj34@liskov> <ZEAh6CewmARbDVuN@telsasoft.com> <20230419192051.xkgyj76akczpbjbs@awork3.anarazel.de> <ZECKP5NwoDGT3+RN@telsasoft.com> <CAAKRu_YFUC+Sh9969RJ7GCSX=nPOQLVJp9UHK_iHtVT=wkBEKQ@mail.gmail.com> <ZEFfVUeNzQaeHy7R@telsasoft.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox