Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1YBsvm-0001KX-6O for pgsql-performance@arkaria.postgresql.org; Thu, 15 Jan 2015 22:30:22 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1YBsvl-0003iG-2p for pgsql-performance@arkaria.postgresql.org; Thu, 15 Jan 2015 22:30:21 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.80) (envelope-from ) id 1YBsvj-0003i6-VK for pgsql-performance@postgresql.org; Thu, 15 Jan 2015 22:30:20 +0000 Received: from mx-out.zmailcloud.com ([192.198.85.98] helo=mx-out-1.zmailcloud.com) by magus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1YBsvf-0005hv-4X for pgsql-performance@postgresql.org; Thu, 15 Jan 2015 22:30:18 +0000 Received: from smtp.01.com (smtp.01.com [10.10.0.43]) by mx-out-1.zmailcloud.com (Postfix) with ESMTP id DAEEA5645C2 for ; Thu, 15 Jan 2015 16:30:11 -0600 (CST) Received: from localhost (localhost [127.0.0.1]) by smtp-out-1.01.com (Postfix) with ESMTP id D0521A02ED for ; Thu, 15 Jan 2015 16:30:11 -0600 (CST) X-Virus-Scanned: amavisd-new at smtp-out-1.01.com Received: from smtp.01.com ([127.0.0.1]) by localhost (smtp-out-1.01.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tYrX3B2JtSKJ for ; Thu, 15 Jan 2015 16:30:11 -0600 (CST) Received: from smtp.01.com (localhost [127.0.0.1]) by smtp-out-1.01.com (Postfix) with ESMTP id A7975A03FD for ; Thu, 15 Jan 2015 16:30:11 -0600 (CST) Received: from localhost (localhost [127.0.0.1]) by smtp-out-1.01.com (Postfix) with ESMTP id 88F7CA03FC for ; Thu, 15 Jan 2015 16:30:11 -0600 (CST) X-Virus-Scanned: amavisd-new at smtp-out-1.01.com Received: from smtp.01.com ([127.0.0.1]) by localhost (smtp-out-1.01.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id AxEHXdl_4tFu for ; Thu, 15 Jan 2015 16:30:11 -0600 (CST) Received: from [172.24.244.13] (wireless-nat-23-linuxconf2015.auckland.ac.nz [130.216.30.134]) by smtp-out-1.01.com (Postfix) with ESMTPSA id 0B513A032B for ; Thu, 15 Jan 2015 16:30:10 -0600 (CST) Message-ID: <54B83F70.7080004@agliodbs.com> Date: Fri, 16 Jan 2015 11:30:08 +1300 From: Josh Berkus Organization: PostgreSQL Experts Inc. User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: postgres performance list Subject: Strange choice of general index over partial index Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org This is an obfuscation and mock up, but: table files ( id serial pk, filename text not null, state varchar(20) not null ... 18 more columns ) index file_state on (state) (35GB in size) index file_in_flight_state (state) where state in ( 'waiting','assigning', 'processing' ) (600MB in size) ... 10 more indexes More important facts: * state = 'done' 95% of the time. thereform the partial index represents only 5% of the table * all indexes and the table are very bloated * server has 128GB RAM * Version 9.2. Given this setup, I would expect the planner to *always* choose file_in_flight_state over file_state for this query: SELECT id, filename FROM files WHERE state = 'waiting'; ... and yet it keeps selecting file_state based on extremely small changes to the stats. This is important because the same query, using file_state, is 20X to 50X slower, because that index frequently gets pushed out of memory. What am I missing? Or is this potentially a planner bug for costing? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance