From: Matt Clark <matt@ymogen.net>
To: Scott Cain <cain@cshl.org>
To: Richard Huxton <dev@archonet.com>
Cc: PgSQL Performance ML <pgsql-performance@postgresql.org>
Cc: pgsql-sql@postgresql.org
Subject: Re: [SQL] EXTERNAL storage and substring on long strings
Date: Mon, 4 Aug 2003 17:56:00 +0100
Message-ID: <OAEAKHEHCMLBLIDGAFELEEDDDGAA.matt@ymogen.net> (raw)
In-Reply-To: <1060013646.1434.40.camel@localhost.localdomain>
> > 2. If you want to search for a sequence you'll need to deal with the case
> > where it starts in one chunk and ends in another.
>
> I forgot about searching--I suspect that application is why I faced
> opposition for shredding in my schema development group. Maybe I should
> push that off to the file system and use grep (or BLAST). Otherwise, I
> could write a function that would search the chunks first, then after
> failing to find the substring in those, I could start sewing the chunks
> together to look for the query string. That could get ugly (and
> slow--but if the user knows that and expects it to be slow, I'm ok with
> that).
If you know the max length of the sequences being searched for, and this is much less than the chunk size, then you could simply
have the chunks overlap by that much, thus guaranteeing every substring will be found in its entirety in at least one chunk.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: matt@ymogen.net, cain@cshl.org, dev@archonet.com, pgsql-performance@postgresql.org
Subject: Re: [SQL] EXTERNAL storage and substring on long strings
In-Reply-To: <OAEAKHEHCMLBLIDGAFELEEDDDGAA.matt@ymogen.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox