public inbox for [email protected]
help / color / mirror / Atom feedFrom: Alena Rybakina <[email protected]>
To: Robert Haas <[email protected]>
To: Alexander Korotkov <[email protected]>
Cc: [email protected]
Cc: Finnerty, Jim <[email protected]>
Cc: Marcos Pegoraro <[email protected]>
Cc: Andrey Lepikhov <[email protected]>
Cc: [email protected]
Cc: Ranier Vilela <[email protected]>
Cc: Tomas Vondra <[email protected]>
Cc: Peter Geoghegan <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Subject: Re: POC, WIP: OR-clause support for indexes
Date: Thu, 26 Oct 2023 22:47:20 +0300
Message-ID: <[email protected]> (raw)
In-Reply-To: <CA+Tgmob=ebuCHFSw327b55DJzE3JtOuZ5owxob+MgErb4me_Ag@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<CAH2-WzmG-+vvN6YEh4J_j3-Q1Yn=HRrjDrz2HDhttPcLVqN0rw@mail.gmail.com>
<[email protected]>
<CAH2-WzkmLg0JPXkZ75b+ha_wJGoJk3QyZ6aUjA0Ba11rdPwdgw@mail.gmail.com>
<[email protected]>
<[email protected]>
<CAH2-Wzkp1RYWC4Lsh==ALNeOR=gQEtCMVbcFRLa-faMQe6YRPA@mail.gmail.com>
<CAH2-Wz=rJOP_1niMTSN_=pzKXUu5tAGRYMGj6uC6Xx=XzkgdPg@mail.gmail.com>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CAPpHfduJtO0s9E=SHUTzrCD88BH0eik0UNog1_q3XBF2wLmH6g@mail.gmail.com>
<CA+Tgmob=ebuCHFSw327b55DJzE3JtOuZ5owxob+MgErb4me_Ag@mail.gmail.com>
Hi! Thank you for your feedback!
On 25.10.2023 22:54, Robert Haas wrote:
> On Sat, Oct 14, 2023 at 6:37 PM Alexander Korotkov<[email protected]> wrote:
>> Regarding the GUC parameter, I don't see we need a limit. It's not
>> yet clear whether a small number or a large number of OR clauses are
>> more favorable for transformation. I propose to have just a boolean
>> enable_or_transformation GUC.
> That's a poor solution. So is the GUC patch currently has
> (or_transform_limit). What you need is a heuristic that figures out
> fairly reliably whether the transformation is going to be better or
> worse. Or else, do the whole thing in some other way that is always
> same-or-better.
>
> In general, adding GUCs makes sense when the user knows something that
> we can't know. For example, shared_buffers makes some sense because,
> even if we discovered how much memory the machine has, we can't know
> how much of it the user wants to devote to PostgreSQL as opposed to
> anything else. And track_io_timing makes sense because we can't know
> whether the user wants to pay the price of gathering that additional
> data. But GUCs are a poor way of handling cases where the real problem
> is that we don't know what code to write. In this case, some queries
> will be better with enable_or_transformation=on, and some will be
> better with enable_or_transformation=off. Since we don't know which
> will work out better, we make the user figure it out and set the GUC,
> possibly differently for each query. That's terrible. It's the query
> optimizer's whole job to figure out which transformations will speed
> up the query. It shouldn't turn around and punt the decision back to
> the user.
>
> Notice that superficially-similar GUCs like enable_seqscan aren't
> really the same thing at all. That's just for developer testing and
> debugging. Nobody expects that you have to adjust that GUC on a
> production system - ever.
I noticed that the costs of expressions are different and it can help to
assess when it is worth leaving the conversion, when not.
With small amounts of "OR" elements, the cost of orexpr is lower than
with "ANY", on the contrary, higher.
postgres=# SET or_transform_limit = 500;
EXPLAIN (analyze)
SELECT oid,relname FROM pg_class
WHERE
oid = 13779 AND (oid = 2 OR oid = 4 OR oid = 5)
;
SET
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
Index Scan using pg_class_oid_index on pg_class (*cost=0.27..8.30*
rows=1 width=68) (actual time=0.105..0.106 rows=0 loops=1)
Index Cond: (oid = '13779'::oid)
Filter: ((oid = '2'::oid) OR (oid = '4'::oid) OR (oid = '5'::oid))
Planning Time: 0.323 ms
Execution Time: 0.160 ms
(5 rows)
postgres=# SET or_transform_limit = 0;
EXPLAIN (analyze)
SELECT oid,relname FROM pg_class
WHERE
oid = 13779 AND (oid = 2 OR oid = 4 OR oid = 5)
;
SET
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------
Index Scan using pg_class_oid_index on pg_class (*cost=0.27..16.86*
rows=1 width=68) (actual time=0.160..0.161 rows=0 loops=1)
Index Cond: ((oid = ANY (ARRAY['2'::oid, '4'::oid, '5'::oid])) AND
(oid = '13779'::oid))
Planning Time: 4.515 ms
Execution Time: 0.313 ms
(4 rows)
Index Scan using pg_class_oid_index on pg_class (*cost=0.27..2859.42*
rows=414 width=68) (actual time=1.504..34.183 rows=260 loops=1)
Index Cond: (oid = ANY (ARRAY['1'::oid, '2'::oid, '3'::oid,
'4'::oid, '5'::oid, '6'::oid, '7'::oid,
Bitmap Heap Scan on pg_class (*cost=43835.00..54202.14* rows=414
width=68) (actual time=39.958..41.293 rows=260 loops=1)
Recheck Cond: ((oid = '1'::oid) OR (oid = '2'::oid) OR (oid =
'3'::oid) OR (oid = '4'::oid) OR (oid =
I think we could see which value is lower, and if lower with expressions
converted to ANY, then work with it further, otherwise work with the
original "OR" expressions. But we still need to make this conversion to
find out its cost.
In addition, I will definitely have to postpone the transformation of
"OR" to "ANY" at the stage of creating indexes (?) or maybe a little
earlier so that I have to count only the cost of the transformed
expression.
view thread (33+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: POC, WIP: OR-clause support for indexes
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox