From: John Gray <jgray@azuli.co.uk>
To: pgsql-general@postgresql.org
Subject: Re: uploading files
Date: Wed, 19 Nov 2003 23:05:38 +0000
Message-ID: <bpgt7u$1fp3$1@news.hub.org> (raw)
References: <1069164075.986.239.camel@bbking.uio.no>
<20031118142919.GI4953@libertyrms.info>
<1069180780.2769.91.camel@haggis.homelan>
<019301c3ae07$d714e220$0700a8c0@trogdor>
<87llqdqy37.fsf@asmodeus.mcnaught.org>
<20031118205057.K606@hermes.hilbert.loc>
<87fzglqvzq.fsf@asmodeus.mcnaught.org>
<20031118212256.L606@hermes.hilbert.loc>
On Tue, 18 Nov 2003 21:22:56 +0100, Karsten Hilbert wrote:
>> I think the field will still be competely loaded into memory on the
>> server side though, while LOs are stored in "chunks" and can
>> theoretically be streamed to the client. I'm not really a definitive
>> authority, though...
> Ah ! Sounds about right ! Something new to learn every day :-)
>
Actually, bytea and text are chunked behind the scenes (the technique
known as TOAST). They are also compressed(LZ) by default. However
if you anticipate substringing them a lot (or if they are not very
compressible anyway), use ALTER TABLE xxx ALTER COLUMN yyy SET STORAGE
EXTERNAL to turn off the automatc compression of values. This allows the
substr operation to read the minimum number of chunks necessary of disk.
Bear in mind of course, that the whole value will take up more disk space,
so reading it in its entirety will be slower (IO is normally the limiting
performance factor -CPU on a DB server is often cheap by comparison, so
decompressing/compressing to save IO is a good idea). If however you
always fetch small parts (e.g. you store large images and usually want to
read the header info from them, EXTERNAL is a good bet (and depending on
the image format, the compression might not compress them very much anyway).
Finally, note that the substr optimisation for text only really buys you
anything if the character-set is single-byte.
Hope this helps
John Gray
(implementer of substr optimisation many moons ago!)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: jgray@azuli.co.uk, pgsql-general@postgresql.org
Subject: Re: uploading files
In-Reply-To: <bpgt7u$1fp3$1@news.hub.org>
References: <1069164075.986.239.camel@bbking.uio.no> <20031118142919.GI4953@libertyrms.info> <1069180780.2769.91.camel@haggis.homelan> <019301c3ae07$d714e220$0700a8c0@trogdor> <87llqdqy37.fsf@asmodeus.mcnaught.org> <20031118205057.K606@hermes.hilbert.loc> <87fzglqvzq.fsf@asmodeus.mcnaught.org> <20031118212256.L606@hermes.hilbert.loc> <bpgt7u$1fp3$1@news.hub.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox