X-Original-To: pgsql-performance@postgresql.org Received: from localhost (developer.postgresql.org [64.117.224.193]) by developer.postgresql.org (Postfix) with ESMTP id BA370925227 for ; Wed, 28 May 2003 09:19:19 -0400 (EDT) Received: from developer.postgresql.org ([64.117.224.193]) by localhost (developer.postgresql.org [64.117.224.193:10024]) (amavisd-new) with ESMTP id 34058-01 for ; Wed, 28 May 2003 09:19:18 -0400 (EDT) Received: from smtp.web.de (smtp03.web.de [217.72.192.158]) by developer.postgresql.org (Postfix) with ESMTP id 0A9AE9251F7 for ; Wed, 28 May 2003 09:19:18 -0400 (EDT) Received: from p508187dc.dip0.t-ipconnect.de ([80.129.135.220] helo=web.de) by smtp.web.de with asmtp (TLSv1:RC4-MD5:128) (WEB.DE 4.98 #232) id 19L0po-0003yV-00; Wed, 28 May 2003 15:19:17 +0200 Message-ID: <3ED4B75A.2050106@web.de> Date: Wed, 28 May 2003 15:19:22 +0200 From: Andreas Pflug User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Dave Tenny , pgsql-performance@postgresql.org Subject: Re: IN list processing performance (yet again) References: <3ED4B0E5.3050703@attbi.com> In-Reply-To: <3ED4B0E5.3050703@attbi.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new X-Archive-Number: 200305/232 X-Sequence-Number: 2011 Dave Tenny wrote: > Having grepped the web, it's clear that this isn't the first or last > time this issue will be raised. > > My application relies heavily on IN lists. The lists are primarily > constant integers, so queries look like: > > SELECT val FROM table WHERE id IN (43, 49, 1001, 100002, ...) > > Performance is critical, and the size of these lists depends a lot on > how the larger 3-tier applicaiton is used, > but it wouldn't be out of the question to retrieve 3000-10000 items. > > PostgreSQL 7.3.2 seems to have a lot of trouble with large lists. > I ran an experiment that ran queries on a table of two integers (ID, > VAL), where ID is a primary key and the subject > of IN list predicates. The test used a table with one million rows > ID is appropriately indexed, > and I have VACUUMED/analyzed the database after table load. > > I ran tests on in-lists from about 100 to 100,000 entries. Hi Dave, it sounds as if that IN-list is created by the application. I wonder if there are really so many variances and combinations of it or whether you could invent an additional column, which groups all those individual values. If possible, you could reduce your IN list to much fewer values, and probably would get better performance (using an index on that col, of course). Regards, Andreas