From: Justin Pryzby <pryzby@telsasoft.com>
To: Matt Dupree <matt.dupree@heap.io>
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: Postgres using the wrong index index
Date: Mon, 23 Aug 2021 20:38:19 -0500
Message-ID: <20210824013819.GK10479@telsasoft.com> (raw)
In-Reply-To: <CAMOk8kpnuDtA7jh9YueU37d39xaQE97rWZE+VvYfdK-susgU1Q@mail.gmail.com>
References: <CAMOk8kqSPr2sTihQ56Va07TeFRA8E1GsEUfwe4t4LSmj5pjd4A@mail.gmail.com>
<20210811123856.GE10479@telsasoft.com>
<CAMOk8kq5Hm80EHTtBR1xD0T0aCDmRoE3nxaWtkNSZqS04CLLfg@mail.gmail.com>
<20210812034510.GG10479@telsasoft.com>
<CAMOk8krqeZieBDgqRPV6HK8nXAra4D29Cc9wUpdyEa=OXH7oOw@mail.gmail.com>
<20210812232006.GN10479@telsasoft.com>
<CAMOk8kqB3X9qFatSJQOubFAW34n8ghR=EPxff20qBnb44n2fKQ@mail.gmail.com>
<20210817185232.GN10479@telsasoft.com>
<CAMOk8kpnuDtA7jh9YueU37d39xaQE97rWZE+VvYfdK-susgU1Q@mail.gmail.com>
On Mon, Aug 23, 2021 at 08:53:15PM -0400, Matt Dupree wrote:
> Is it possible that the row estimate is off because of a column other than
> time?
I would test this by writing the simplest query that reproduces the
mis-estimate.
> I looked at the # of events in that time period and 1.8 million is
> actually a good estimate. What about the
> ((strpos(other_events_1004175222.hierarchy, '#close_onborading;'::text) <>
> 0) condition in the filter? It makes sense that Postgres wouldn't have a
> way to estimate how selective this condition is.
The issue I see is here. I don't know where else I'd start but to understand
this.
| Index Scan using other_events_1004175222_pim_evdef_67951aef14bc_idx on public.other_events_1004175222 (cost=0.28..1,648,877.92 ROWS=1,858,891 width=32) (actual time=1.008..15.245 ROWS=23 loops=1)
| Output: other_events_1004175222.user_id, other_events_1004175222."time", other_events_1004175222.session_id
| Index Cond: ((other_events_1004175222."time" >= '1624777200000'::bigint) AND (other_events_1004175222."time" <= '1627369200000'::bigint))
| Buffers: shared read=25
This has no "filter" condition, it's a "scan" node with bad over-estimate.
Note that this is due to the table's column stats, not any index's stats, so
every plan is affected. even though some happen to work well. The consequences
of over-estimates are not as terrible as for under-estimates, but it's bad to
start with inputs that are off by 10^5.
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: pryzby@telsasoft.com, matt.dupree@heap.io, pgsql-performance@lists.postgresql.org
Subject: Re: Postgres using the wrong index index
In-Reply-To: <20210824013819.GK10479@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox