From: Jan Wieck <JanWieck@Yahoo.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Scott Cain <cain@cshl.org>
Cc: PgSQL Performance ML <pgsql-performance@postgresql.org>
Cc: pgsql-sql@postgresql.org
Subject: Re: [SQL] EXTERNAL storage and substring on long strings
Date: Thu, 14 Aug 2003 11:02:53 -0400
Message-ID: <3F3BA49D.9070405@Yahoo.com> (raw)
In-Reply-To: <3574.1060012423@sss.pgh.pa.us>
References: <1059679600.1429.41.camel@localhost.localdomain>
<3F297CBA.1080703@joeconway.com>
<1059686498.7505.558.camel@haggis>
<1060010735.1433.26.camel@localhost.localdomain>
<3574.1060012423@sss.pgh.pa.us>
Tom Lane wrote:
> Scott Cain <cain@cshl.org> writes:
>> At least this appears to work and is much faster, completing substring
>> operations like above in about 0.27 secs (that's about two orders of
>> magnitude improvement!)
>
> I find it really, really hard to believe that a crude reimplementation
> in plpgsql of the TOAST concept could beat the built-in implementation
> at all, let alone beat it by two orders of magnitude.
>
> Either there's something unrealistic about your testing of the
> dna_string function, or your original tests are not causing TOAST to be
> invoked in the expected way, or there's a bug we need to fix. I'd
> really like to see some profiling of the poor-performing
> external-storage case, so we can figure out what's going on.
Doesn't look that unrealistic to me. A plain text based substring
function will reassemble the whole beast first before cutting out the
wanted part. His manually chunked version will read only those chunks
needed. Considering that he's talking about retrieving a few thousand
chars from a hundreds of MB size string ...
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: JanWieck@Yahoo.com, tgl@sss.pgh.pa.us, cain@cshl.org, pgsql-performance@postgresql.org
Subject: Re: [SQL] EXTERNAL storage and substring on long strings
In-Reply-To: <3F3BA49D.9070405@Yahoo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox