pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Mark Mielke <mark@mark.mielke.cc>
Cc: Matthew Wakeling <matthew@flymine.org>
Cc: pgsql-performance@postgresql.org
Subject: Re: RAID arrays and performance
Date: Fri, 19 Sep 2008 12:22:51 -0400
Message-ID: <48D3D1DB.5010803@mark.mielke.cc> (raw)
In-Reply-To: <21255.1221836996@sss.pgh.pa.us>
References: <Pine.LNX.4.58.0712041213490.3731@aragorn.flymine.org>
	<87lk8aiqf3.fsf@oxford.xeocode.com>
	<Pine.LNX.4.58.0712041247540.3731@aragorn.flymine.org>
	<475559F4.7060104@mark.mielke.cc>
	<Pine.LNX.4.58.0712041349380.3731@aragorn.flymine.org>
	<4755648C.90905@mark.mielke.cc>
	<Pine.LNX.4.58.0712041454110.3731@aragorn.flymine.org>
	<87bq96ii3b.fsf@oxford.xeocode.com>
	<Pine.LNX.4.58.0712041603490.3731@aragorn.flymine.org>
	<4755810E.4000707@mark.mielke.cc>
	<Pine.LNX.4.58.0712041637260.3731@aragorn.flymine.org>
	<87zlwq6yz9.fsf@oxford.xeocode.com>
	<Pine.LNX.4.64.0801291417330.4642@aragorn.flymine.org>
	<87bq74vfys.fsf@oxford.xeocode.com>
	<alpine.DEB.1.10.0809181417040.23198@aragorn.flymine.org>
	<Pine.GSO.4.64.0809181515380.7355@westnet.com>
	<alpine.DEB.1.10.0809191535050.23198@aragorn.flymine.org>
	<21255.1221836996@sss.pgh.pa.us>

Matthew Wakeling <matthew@flymine.org> writes:
>> In order to improve the performance, I made the system look ahead in the 
>> source, in groups of a thousand entries, so instead of running:
>> SELECT * FROM table WHERE field = 'something';
>> a thousand times, we now run:
>> SELECT * FROM table WHERE field IN ('something', 'something else'...);
>> with a thousand things in the IN. Very simple query. It does run faster 
>> than the individual queries, but it still takes quite a while. Here is an 
>> example query:
>>     

Have you considered temporary tables? Use COPY to put everything you 
want to query into a temporary table, then SELECT to join the results, 
and pull all of the results, doing additional processing (UPDATE) as you 
pull?

Cheers,
mark

-- 
Mark Mielke <mark@mielke.cc>

view thread (39+ messages)

Message-ID: <48D3D1DB.5010803@mark.mielke.cc>
Permalink:  ../48D3D1DB.5010803@mark.mielke.cc/
Also on:    postgresql.org/message-id/48D3D1DB.5010803@mark.mielke.cc

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: mark@mark.mielke.cc, matthew@flymine.org
  Subject: Re: RAID arrays and performance
  In-Reply-To: <48D3D1DB.5010803@mark.mielke.cc>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox