Date: Mon, 8 May 2006 12:50:13 -0500
From: "Jim C. Nasby" <jnasby@pervasive.com>
To: Mark Lewis <mark.lewis@mir3.com>
Cc: Jeffrey Tenny <jeffrey.tenny@comcast.net>,
 pgsql-performance@postgresql.org
Subject: Re: performance question (something to do w/
Message-ID: <20060508175013.GY99570@pervasive.com>
References: <445F7FF8.6070707@comcast.net>
	<1147110142.32369.7.camel@archimedes>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1147110142.32369.7.camel@archimedes>
User-Agent: Mutt/1.5.11

On Mon, May 08, 2006 at 10:42:21AM -0700, Mark Lewis wrote:
> Doing a SELECT with a large list of variables inside an IN runs slowly
> on every database we've tested.  We've tested mostly in Oracle and
> PostgreSQL, and both get very slow very quickly (actually Oracle refuses
> to process the query at all after it gets too many bind parameters).
> 
> In our case, we have a (potentially quite large) set of external values
> that we want to look up in the database.  We originally thought that
> doing a single select with a large IN clause was the way to go, but then
> we did some performance analysis on the optimal batch size (number of
> items to include per IN clause), and discovered that for most databases,
> the optimal batch size was 1.  For PostgreSQL I think it was 2.
> 
> The moral of the story is that you're probably better off running a
> bunch of small selects than in trying to optimize things with one
> gargantuan select.

Ever experiment with loading the parameters into a temp table and
joining to that?
-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461