Subject: Re: EXTERNAL storage and substring on long strings
From: Ron Johnson <ron.l.johnson@cox.net>
To: PgSQL Performance ML <pgsql-performance@postgresql.org>
In-Reply-To: <3F297CBA.1080703@joeconway.com>
References: <1059679600.1429.41.camel@localhost.localdomain>
	<3F297CBA.1080703@joeconway.com>
Content-Type: text/plain
Message-Id: <1059686498.7505.558.camel@haggis>
Mime-Version: 1.0
Date: 31 Jul 2003 16:21:38 -0500
Content-Transfer-Encoding: 7bit

On Thu, 2003-07-31 at 15:31, Joe Conway wrote:
> Scott Cain wrote:
> >  Index Scan using feature_pkey on feature  (cost=0.00..3.01 rows=1
> > width=153) (actual time=954.13..954.14 rows=1 loops=1)
> >    Index Cond: (feature_id = 1)
> >  Total runtime: 954.26 msec
> > (3 rows)
> > 
> > Whoa!  That's not what I expected, the time to do the query got more
> > that twice as long.  So I think, maybe it was just an unlucky section,
> > and overall performance will be much better.  So I write a perl script
> > to do substring queries over all of my chromosomes at various positions
> > and lengths (20,000 queries total).  For comparison, I also ran the same
> > script, extracting the chromosomes via sql and doing the substring in
> > perl.  Here's what happened:
> 
> Hmmm, what happens if you compare with a shorter substring, e.g.:
> 
> explain analyze select substring(residues from 1000000 for 2000)
> from feature where feature_id=1;
> 
> I'm just guessing, but it might be that the extra I/O time to read 20K 
> of uncompressed text versus the smaller compressed text is enough to 
> swamp the time saved from not needing to uncompress.

Are you asking, "Can his CPU decompress faster than his disks can
read?"

-- 
+-----------------------------------------------------------------+
| Ron Johnson, Jr.        Home: ron.l.johnson@cox.net             |
| Jefferson, LA  USA                                              |
|                                                                 |
| "I'm not a vegetarian because I love animals, I'm a vegetarian  |
|  because I hate vegetables!"                                    |
|    unknown                                                      |
+-----------------------------------------------------------------+