Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pGjKJ-0002EW-IS for pgsql-general@arkaria.postgresql.org; Sat, 14 Jan 2023 16:24:15 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pGjJK-0002QM-A8 for pgsql-general@arkaria.postgresql.org; Sat, 14 Jan 2023 16:23:14 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pGjJJ-0002QC-Vp for pgsql-general@lists.postgresql.org; Sat, 14 Jan 2023 16:23:13 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pGjJH-00075r-FR for pgsql-general@lists.postgresql.org; Sat, 14 Jan 2023 16:23:13 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 30EGN7o71008140; Sat, 14 Jan 2023 11:23:07 -0500 From: Tom Lane To: Harmen cc: pgsql-general@lists.postgresql.org Subject: Re: row estimate for partial index In-reply-to: <20230113150159.GJ91568@arp.lijzij.de> References: <20230113150159.GJ91568@arp.lijzij.de> Comments: In-reply-to Harmen message dated "Fri, 13 Jan 2023 16:01:59 +0100" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <1008138.1673713387.1@sss.pgh.pa.us> Content-Transfer-Encoding: quoted-printable Date: Sat, 14 Jan 2023 11:23:07 -0500 Message-ID: <1008139.1673713387@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Harmen writes: > Works well enough. However, we now have an org_id which has > 10% of the= rows, > but only a handful rows where "deleted is null" matches (so the org has = a lot > of "deleted" contacts). The planner doesn't like this and it falls back = to a > full table scan for the above query. > I've added a dedicated index just for that org_id, to see if that helps: > "org123" btree (id) WHERE deleted IS NULL AND org_id =3D 123 > The planner seems to use it now, however the row estimate is way off: Yeah, so that indicates that it isn't producing a good selectivity estimate for the combination of those two conditions: it will assume the org_id and deleted columns are independent, which per your statements they are not. If you are running a reasonably recent PG version you should be able to fix that by setting up "extended statistics" on that pair of columns: https://www.postgresql.org/docs/current/planner-stats.html#PLANNER-STATS-E= XTENDED (I might be wrong, but I think that will help even when one of the troublesome conditions is a null-check. If it doesn't, then we have something to improve there ...) regards, tom lane