Message-ID: <54B85D32.7090106@catalyst.net.nz>
Date: Fri, 16 Jan 2015 13:37:06 +1300
From: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Thunderbird/31.3.0
MIME-Version: 1.0
To: Josh Berkus <josh@agliodbs.com>,
 postgres performance list <pgsql-performance@postgresql.org>
Subject: Re: Strange choice of general index over partial index
References: <54B83F70.7080004@agliodbs.com>
In-Reply-To: <54B83F70.7080004@agliodbs.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

On 16/01/15 11:30, Josh Berkus wrote:
> This is an obfuscation and mock up, but:
>
> table files (
> 	id serial pk,
> 	filename text not null,
> 	state varchar(20) not null
> 	... 18 more columns
> )
>
> index file_state on (state)
> 	(35GB in size)
> index file_in_flight_state (state) where state in (
> 'waiting','assigning', 'processing' )
> 	(600MB in size)
> ... 10 more indexes
>
> More important facts:
> * state = 'done' 95% of the time.  thereform the partial index
> represents only 5% of the table
> * all indexes and the table are very bloated
> * server has 128GB RAM
> * Version 9.2.
>
> Given this setup, I would expect the planner to *always* choose
> file_in_flight_state over file_state for this query:
>
> SELECT id, filename FROM files WHERE state = 'waiting';
>
> ... and yet it keeps selecting file_state based on extremely small
> changes to the stats.   This is important because the same query, using
> file_state, is 20X to 50X slower, because that index frequently gets
> pushed out of memory.
>
> What am I missing?  Or is this potentially a planner bug for costing?
>

Are you seeing a bitmapscan access plan? If so see if disabling it gets 
you a plan on the files_in_flight index. I'm seeing this scenario with a 
fake/generated dataset a bit like yours in 9.2 (9.5 uses the 
files_in_flight w/o any coercing).

regards

Mark


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance