pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Daniel Gustafsson <daniel@yesql.se>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Manuel Rigger <rigger.manuel@gmail.com>
Cc: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: Failed assertion clauses != NIL
Date: Tue, 26 Nov 2019 18:41:31 +0100
Message-ID: <20191126174131.p3zv5t2oeqkinx4d@development> (raw)
In-Reply-To: <CAEZATCXccM8m8r=g+UhS3K9LUFNONVf8WvmWJwGcpJSJ-km-6A@mail.gmail.com>
References: <CA+u7OA7H5rcE2=8f263w4NZD6ipO_XOrYB816nuLXbmSTH9pQQ@mail.gmail.com>
	<20191119133839.5qj5zia4pkd3zldt@localhost>
	<520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se>
	<20191119150840.nmezhsf2tpip7osh@development>
	<CAEZATCXXF82RXJzUZATPo3OsKn137ojxNSz83LgsxhDhzVmVjg@mail.gmail.com>
	<20191121145308.zzdcs5uozhtlmcos@development>
	<20191124234011.iq7wnq2gohdcsroj@development>
	<CAEZATCXccM8m8r=g+UhS3K9LUFNONVf8WvmWJwGcpJSJ-km-6A@mail.gmail.com>

On Tue, Nov 26, 2019 at 10:56:33AM +0000, Dean Rasheed wrote:
>On Sun, 24 Nov 2019 at 23:40, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
>>
>> Hi,
>>
>> Attached are two patched, related to this bug report.
>>
>>
>> 0001 - Fix choose_best_statistics to check clauses individually
>> ---------------------------------------------------------------
>>
>> This modifies the choose_best_statistics function to properly check
>> which clauses are actually covered by each statistic object, and only
>> use attnums from those.
>>
>> The patch ended up pretty small, because we already have all the
>> necessary info (per-clause attnums) precalculated. Which means this
>> should not be much more expensive than before.
>>
>> The main drawback is that this does change signature of a function
>> defined in statistics.h - we have to pass more info (per-clause bitmaps
>> and info which clauses are already estimated). Which means ABI break.
>>
>> I'm not sure how likely it is that external code is calling this
>> function, but the probability is non-zero. So maybe even if the patch is
>> fairly small, in backbranches we should use the simple fix with just
>> returning if the list is NIL.
>>
>
>On a quick read-through that algorithm makes a lot more sense. It
>seems pretty unlikely that anyone would be using
>choose_best_statistics() anywhere else, so I think maybe it's fine to
>change that in back-branches.
>
>A couple of comments:
>
>1). I think you should pass estimatedClauses to
>choose_best_statistics() as a pure input parameter (i.e., remove one
>"*"), since it doesn't (and must not) modify that set.

Yeah, good point.

>In fact, on closer inspection, I don't think you need to pass it to
>choose_best_statistics() at all, since its callers already check
>clauses against estimatedClauses. Therefore, in
>choose_best_statistics(), incompatible and already-estimated clauses
>both appear the same (as NULL/empty attribute sets), and therefore the
>estimatedClauses check will never be tripped.
>

Right, but I'm thinking about the patch that allows applying multiple
statistics. With that applied, this changes to a while loop - and we'll
either have to rebuild the list_attnums or pass the bitmap.

>2). The new parameter "clauses" should probably be called something
>like "clause_attnums" or some such, to better reflect what it actually
>is. And it should be documented that NULL values represent
>incompatible/already-estimated clauses.
>

Yes, agreed.

>3). In statext_mcv_clauselist_selectivity(), the check for at least 2
>attributes is no longer needed, because choose_best_statistics() now
>does that, so there's also no need to compute clauses_attnums there.
>

Good point. I'll replace that with an assert.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services





view thread (13+ messages)  latest in thread

Message-ID: <20191126174131.p3zv5t2oeqkinx4d@development>
Permalink:  ../20191126174131.p3zv5t2oeqkinx4d@development/
Also on:    postgresql.org/message-id/20191126174131.p3zv5t2oeqkinx4d@development

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, dean.a.rasheed@gmail.com, daniel@yesql.se, 9erthalion6@gmail.com, rigger.manuel@gmail.com
  Subject: Re: Failed assertion clauses != NIL
  In-Reply-To: <20191126174131.p3zv5t2oeqkinx4d@development>
  References: <CA+u7OA7H5rcE2=8f263w4NZD6ipO_XOrYB816nuLXbmSTH9pQQ@mail.gmail.com> <20191119133839.5qj5zia4pkd3zldt@localhost> <520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se> <20191119150840.nmezhsf2tpip7osh@development> <CAEZATCXXF82RXJzUZATPo3OsKn137ojxNSz83LgsxhDhzVmVjg@mail.gmail.com> <20191121145308.zzdcs5uozhtlmcos@development> <20191124234011.iq7wnq2gohdcsroj@development> <CAEZATCXccM8m8r=g+UhS3K9LUFNONVf8WvmWJwGcpJSJ-km-6A@mail.gmail.com> <20191126174131.p3zv5t2oeqkinx4d@development>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox