pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedRe: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug)
4+ messages / 2 participants
[nested] [flat]
* Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug)
@ 2018-02-06 18:18 Justin Pryzby <pryzby@telsasoft.com>
2018-02-06 20:02 ` Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Rick Otten <rottenwindfish@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Justin Pryzby @ 2018-02-06 18:18 UTC (permalink / raw)
To: Rick Otten <rottenwindfish@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-performance
On Sun, Feb 04, 2018 at 11:04:56AM -0500, Rick Otten wrote:
> On Sun, Feb 4, 2018 at 10:35 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> > Rick Otten <rottenwindfish@gmail.com> writes:
> > > I'm wrestling with a very similar problem too - except instead of
> > official
> > > partitions I have a views on top of a bunch (50+) of unioned materialized
> > > views, each "partition" with 10M - 100M rows. On 9.6.6 the queries would
> > > use the indexes on each materialized view. On 10.1, every materialized
> > > view is sequence scanned.
I think it'd be useful to see the plan from explain analyze, on both the
"parent" view and a child, with and without SET enable_seqscan=off,
Justin
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug)
2018-02-06 18:18 Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Justin Pryzby <pryzby@telsasoft.com>
@ 2018-02-06 20:02 ` Rick Otten <rottenwindfish@gmail.com>
2018-02-06 20:03 ` Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Rick Otten <rottenwindfish@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Rick Otten @ 2018-02-06 20:02 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-performance
Ooo. I wasn't aware of that option. (Learn something new every day!)
Setting enable_seqscan=off takes one of the shorter queries I was working
with from about 3 minutes to 300ms. This is a comparable performance
improvement to where I put a materialized view (with indexes) on top of the
materialized views instead of using a simple view on top of the
materialized views. I'll have to try it with the query that takes 12 hours.
I built a test case, but can't get it to reproduce what I'm seeing on my
production database (it keeps choosing the indexes). I'm still fiddling
with that test case so I can easily share it. I'm also back to trying to
figure out what is different between my laptop database and the test case I
built and the real world query with the real data, and pondering the worst
query itself to see if some sort of re-write will help.
On Tue, Feb 6, 2018 at 1:18 PM, Justin Pryzby <pryzby@telsasoft.com> wrote:
> On Sun, Feb 04, 2018 at 11:04:56AM -0500, Rick Otten wrote:
> > On Sun, Feb 4, 2018 at 10:35 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > > Rick Otten <rottenwindfish@gmail.com> writes:
> > > > I'm wrestling with a very similar problem too - except instead of
> > > official
> > > > partitions I have a views on top of a bunch (50+) of unioned
> materialized
> > > > views, each "partition" with 10M - 100M rows. On 9.6.6 the queries
> would
> > > > use the indexes on each materialized view. On 10.1, every
> materialized
> > > > view is sequence scanned.
>
> I think it'd be useful to see the plan from explain analyze, on both the
> "parent" view and a child, with and without SET enable_seqscan=off,
>
> Justin
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug)
2018-02-06 18:18 Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Justin Pryzby <pryzby@telsasoft.com>
2018-02-06 20:02 ` Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Rick Otten <rottenwindfish@gmail.com>
@ 2018-02-06 20:03 ` Rick Otten <rottenwindfish@gmail.com>
2018-02-08 11:04 ` Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Rick Otten <rottenwindfish@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Rick Otten @ 2018-02-06 20:03 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-performance
On Tue, Feb 6, 2018 at 3:02 PM, Rick Otten <rottenwindfish@gmail.com> wrote:
> Ooo. I wasn't aware of that option. (Learn something new every day!)
>
> Setting enable_seqscan=off takes one of the shorter queries I was working
> with from about 3 minutes to 300ms. This is a comparable performance
> improvement to where I put a materialized view (with indexes) on top of the
> materialized views instead of using a simple view on top of the
> materialized views. I'll have to try it with the query that takes 12 hours.
>
> I built a test case, but can't get it to reproduce what I'm seeing on my
> production database (it keeps choosing the indexes). I'm still fiddling
> with that test case so I can easily share it. I'm also back to trying to
> figure out what is different between my laptop database and the test case I
> built and the real world query with the real data, and pondering the worst
> query itself to see if some sort of re-write will help.
>
>
>
> On Tue, Feb 6, 2018 at 1:18 PM, Justin Pryzby <pryzby@telsasoft.com>
> wrote:
>
>> On Sun, Feb 04, 2018 at 11:04:56AM -0500, Rick Otten wrote:
>> > On Sun, Feb 4, 2018 at 10:35 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> >
>> > > Rick Otten <rottenwindfish@gmail.com> writes:
>> > > > I'm wrestling with a very similar problem too - except instead of
>> > > official
>> > > > partitions I have a views on top of a bunch (50+) of unioned
>> materialized
>> > > > views, each "partition" with 10M - 100M rows. On 9.6.6 the queries
>> would
>> > > > use the indexes on each materialized view. On 10.1, every
>> materialized
>> > > > view is sequence scanned.
>>
>> I think it'd be useful to see the plan from explain analyze, on both the
>> "parent" view and a child, with and without SET enable_seqscan=off,
>>
>> Justin
>>
>
>
Sorry, I didn't mean to "top reply". My bad.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug)
2018-02-06 18:18 Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Justin Pryzby <pryzby@telsasoft.com>
2018-02-06 20:02 ` Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Rick Otten <rottenwindfish@gmail.com>
2018-02-06 20:03 ` Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Rick Otten <rottenwindfish@gmail.com>
@ 2018-02-08 11:04 ` Rick Otten <rottenwindfish@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Rick Otten @ 2018-02-08 11:04 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-performance
>
>
>>
>> Setting enable_seqscan=off takes one of the shorter queries I was working
>> with from about 3 minutes to 300ms. This is a comparable performance
>> improvement to where I put a materialized view (with indexes) on top of the
>> materialized views instead of using a simple view on top of the
>> materialized views. I'll have to try it with the query that takes 12 hours.
>>
>>
>
The query that takes 12 hours and won't use indexes when I feel it should
is a materialized view refresh. When I set it before testing the plan with
a simple explain on the query it definitely gets it to use all of the
indexes. Does setting something like "enable_seqscan=off" work when I
follow it with a "refresh materialized view concurrently" instead of a
simple select? I'll try it to see if it helps the refresh time, but I
thought I'd ask.
(I got pulled into another problem since my last email, so I haven't had a
chance to follow up.)
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2018-02-08 11:04 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-02-06 18:18 Re: failing to use index on UNION of matviews (Re: postgresql 10.1 wrong plan in when using partitions bug) Justin Pryzby <pryzby@telsasoft.com>
2018-02-06 20:02 ` Rick Otten <rottenwindfish@gmail.com>
2018-02-06 20:03 ` Rick Otten <rottenwindfish@gmail.com>
2018-02-08 11:04 ` Rick Otten <rottenwindfish@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox