Date: Wed, 20 Feb 2019 02:05:00 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Lewis <mlewis@entrata.com>
Cc: Andrew Gierth <andrew@tao11.riddles.org.uk>, Abi Noda <a@abinoda.com>,
	pgsql-performance@lists.postgresql.org
Subject: Re: Why isn't an index scan being used?
Message-ID: <20190220080500.GO28750@telsasoft.com>
References: 
 <CAM37AMPQjpDWNy58W442FSBLEJwLJGgp=xmZts=mj2QzEz-CFw@mail.gmail.com>
 <87ftsji2tb.fsf@news-spur.riddles.org.uk>
 <CAHOFxGowsSLFNLAwpQ5gFPw4F1nT19gNLHRnav0fNL6sykCZbQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAHOFxGowsSLFNLAwpQ5gFPw4F1nT19gNLHRnav0fNL6sykCZbQ@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Precedence: bulk

On Tue, Feb 19, 2019 at 09:29:46PM -0700, Michael Lewis wrote:
> On Tue, Feb 19, 2019, 8:00 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
> 
> > >>>>> "Abi" == Abi Noda <a@abinoda.com> writes:
> >  Abi> However, when I index the closed column, a bitmap scan is used
> >  Abi> instead of an index scan, with slightly slower performance. Why
> >  Abi> isn't an index scan being used, given that the exact same number
> >  Abi> of rows are at play as in my query on the state column?
> >
> > Most likely difference is the correlation estimate for the conditions.
> > The cost of an index scan includes a factor based on how well correlated
> > the physical position of rows is with the index order, because this
> > affects the number of random seeks in the scan. But for nulls this
> > estimate cannot be performed, and bitmapscan is cheaper than plain
> > indexscan on poorly correlated data.
> 
> Does this imply that the optimizer would always prefer the bitmapscan
> rather than index scan even if random page cost = 1, aka sequential cost,
> when the correlation is unknown like a null? Or only when it thinks random
> access is more expensive by some significant factor?

No; for one, since for a bitmap scan, the heap scan can't begin until the index
scan is done, so there's a high(er) initial cost.

Otherwise bitmap scan could always be used and all access could be ordered
(even if not sequential).

Justin