From: wieck@debis.com
To: mitch@venux.net
Cc: pgsql-sql@postgreSQL.org
Subject: Re: [SQL] Dilema.
Date: Sat, 11 Dec 1999 01:03:35 +0100 (MET)
Message-ID: <m11wa0d-0003kGC@orion.SAPserv.Hamburg.dsh.de> (raw)
In-Reply-To: <00a201bf4356$a1640b80$0300000a@doot.org>
> I have a problem that I'm hoping someone can help me with. I've built a
> web-pased application that is back-ended by PGsql.. It works great, I
> couldn't ask for better preformance.
And Bruce just said (on the hackers list) that big tuples
aren't an item of big pressure. Reading this less than 24
hours past tells another story.
> However, I just recently coded another module for it and this module needed
> to accept text files (or just large text blocks lets say). I ran into the 8k
> query limit barrier. I need these blocks of text to be searchable and am
> having to do this with a flat-file based solution right now, it's slow and
> cumbersome. I would much rather have the text stored in a PG field so I
> don't have to stray from the design of the search engine (which is designed
> to search only the pg database, but has been hacked to search flat files
> too).. I could break the texts up but they really all need to be in a single
> field, not broken up. I can't think up a way to do it with the 8k query
> limit.
If you really need to store big texts into a table, a
combination of a view, update rules and PL functions can
definitely do it. It might look a little complex first, but
after understanding the trick, it's pretty neat.
Look at this:
pgsql=> \d bttest
View "bttest"
Attribute | Type | Extra
-----------+------+-------
key | int4 |
content | text |
View definition: SELECT bttest_shadow."key",
bigtext_fetch(bttest_shadow.btid) AS content FROM
bttest_shadow;
pgsql=> select key, length(content) from bttest;
key | length
-----+--------
1 | 3
2 | 9
4 | 135840
(3 rows)
Yes, the row with key=4 really contains a text of that size.
And I can get it back exactly as it was inserted. And yes, I
can do INSERT/UPDATE/DELETE on bttest. Anything behaves as if
it where a regular table.
But it needs the unlimited querysize, supported only in
CURRENT development tree, so you'll have to wait for 7.0
anyway.
> Also feel free to suggest some kind of indexing system for the flat files,
> the only problem with that will be integration into my existing application
> but I've left some hooks open so I can deal with it..
That's another story, and Tom's comment on it says it all.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-sql@postgresql.org
Cc: wieck@debis.com, mitch@venux.net
Subject: Re: [SQL] Dilema.
In-Reply-To: <m11wa0d-0003kGC@orion.SAPserv.Hamburg.dsh.de>
References: <00a201bf4356$a1640b80$0300000a@doot.org> <m11wa0d-0003kGC@orion.SAPserv.Hamburg.dsh.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox