pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Lewis <mlewis@entrata.com>
Cc: Marco Colli <collimarco91@gmail.com>
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: Slow "not in array" operation
Date: Tue, 12 Nov 2019 13:53:34 -0600
Message-ID: <20191112195334.GP2923@telsasoft.com> (raw)
In-Reply-To: <CAHOFxGpt73epD-gYH32u9pOZJkvEO=iNstGJ+jkSkd59LuEtMQ@mail.gmail.com>
References: <CAFvCgN73NJbSdtCFvcD3=Hh+xGipDkUk+H4uzubQXx2rncQJ+w@mail.gmail.com>
	<CAHOFxGq3Udhpd9Y49JwCzqR_bUjCDzGsMX36V57ryzpbaG1hdg@mail.gmail.com>
	<CAFvCgN6bQhf03jdakdk1eVMad8r=6nFV=cmy2v_sp0f8_kZ-bg@mail.gmail.com>
	<CAHOFxGpt73epD-gYH32u9pOZJkvEO=iNstGJ+jkSkd59LuEtMQ@mail.gmail.com>

On Tue, Nov 12, 2019 at 12:20:10PM -0700, Michael Lewis wrote:
> It is very interesting to me that the optimizer chose a parallel sequential
> scan rather than an index scan on either of your indexes that start
> with project_id that also reference trashed_at.

Maybe because of low correlation on any of those columns?
https://wiki.postgresql.org/wiki/Slow_Query_Questions#Statistics:_n_distinct.2C_MCV.2C_histogram
SELECT (SELECT sum(x) FROM unnest(most_common_freqs) x) frac_MCV, tablename, attname, inherited, null_frac, n_distinct, array_length(most_common_vals,1) n_mcv, array_length(histogram_bounds,1) n_hist, correlation FROM pg_stats WHERE tablename='subscriptions' AND  attname IN ('project_id','tags') ORDER BY 1 DESC; 

Maybe clustering the table on project_id (and ANALYZEing) would help, but that
might need to be done consistently.

Michael previously suggested partitioning which, if done on project_id,
would then no longer need to be specially CLUSTERed.

Is the plan for the fast query the same as in August ?
https://www.postgresql.org/message-id/CAFvCgN4UijKTYiOF61Tyd%2BgHvF_oqnMabatS9%2BDcX%2B_PK2SHRw%40ma...

Justin





view thread (17+ messages)  latest in thread

Message-ID: <20191112195334.GP2923@telsasoft.com>
Permalink:  ../20191112195334.GP2923@telsasoft.com/
Also on:    postgresql.org/message-id/20191112195334.GP2923@telsasoft.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: pryzby@telsasoft.com, mlewis@entrata.com, collimarco91@gmail.com, pgsql-performance@lists.postgresql.org
  Subject: Re: Slow "not in array" operation
  In-Reply-To: <20191112195334.GP2923@telsasoft.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox