From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Daniel Gustafsson <daniel@yesql.se>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Manuel Rigger <rigger.manuel@gmail.com>
Cc: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: Failed assertion clauses != NIL
Date: Mon, 25 Nov 2019 00:40:11 +0100
Message-ID: <20191124234011.iq7wnq2gohdcsroj@development> (raw)
In-Reply-To: <20191121145308.zzdcs5uozhtlmcos@development>
References: <CA+u7OA7H5rcE2=8f263w4NZD6ipO_XOrYB816nuLXbmSTH9pQQ@mail.gmail.com>
<20191119133839.5qj5zia4pkd3zldt@localhost>
<520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se>
<20191119150840.nmezhsf2tpip7osh@development>
<CAEZATCXXF82RXJzUZATPo3OsKn137ojxNSz83LgsxhDhzVmVjg@mail.gmail.com>
<20191121145308.zzdcs5uozhtlmcos@development>
Hi,
Attached are two patched, related to this bug report.
0001 - Fix choose_best_statistics to check clauses individually
---------------------------------------------------------------
This modifies the choose_best_statistics function to properly check
which clauses are actually covered by each statistic object, and only
use attnums from those.
The patch ended up pretty small, because we already have all the
necessary info (per-clause attnums) precalculated. Which means this
should not be much more expensive than before.
The main drawback is that this does change signature of a function
defined in statistics.h - we have to pass more info (per-clause bitmaps
and info which clauses are already estimated). Which means ABI break.
I'm not sure how likely it is that external code is calling this
function, but the probability is non-zero. So maybe even if the patch is
fairly small, in backbranches we should use the simple fix with just
returning if the list is NIL.
0002 - WIP: Use extended statistics to estimate OR clauses
----------------------------------------------------------
No matter what 0001 does, it's clear the current code fails to handle OR
clauses that are not fully covered by an extended statistic. For AND
clauses that's not an issue - we simply estimate the covered ones, and
then add the remaining ones by assuming independence.
But clauselist_selectivity sees OR clauses as a single single clause,
and clause_selectivity() simply used the (s1+s2-s1*s2) formula without
considering extended statistics for is_orclause. (For is_andclause we
call clauselist_selectivity recursively, which does consider extended
stats, of course.)
This commit addresses this by calling a clauselist_selectivity variant
for clauses connected by OR, and calling it from the is_orclause branch.
It then requires a bunch of changes elsewhere, to propagate the is_or
flag properly etc.
This is clearly not a thing we could/want to backpatch, and at this
point it's not anywhere close to committable. It's more a WIP patch
highlighting places that will need tweaking to make this work.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
0001-Fix-choose_best_statistics-to-check-clauses-individu.patch.gz (application/gzip, 2.8K) download
0002-WIP-Use-extended-statistics-to-estimate-OR-clauses.patch.gz (application/gzip, 2.9K) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, dean.a.rasheed@gmail.com, daniel@yesql.se, 9erthalion6@gmail.com, rigger.manuel@gmail.com
Subject: Re: Failed assertion clauses != NIL
In-Reply-To: <20191124234011.iq7wnq2gohdcsroj@development>
References: <CA+u7OA7H5rcE2=8f263w4NZD6ipO_XOrYB816nuLXbmSTH9pQQ@mail.gmail.com> <20191119133839.5qj5zia4pkd3zldt@localhost> <520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se> <20191119150840.nmezhsf2tpip7osh@development> <CAEZATCXXF82RXJzUZATPo3OsKn137ojxNSz83LgsxhDhzVmVjg@mail.gmail.com> <20191121145308.zzdcs5uozhtlmcos@development> <20191124234011.iq7wnq2gohdcsroj@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox