From: wieck@debis.com
To: Philippe Gobin <philippe.gobin@fth.net>
Cc: pgsql-sql@postgresql.org
Subject: TOAST (was: BLOB)
Date: Wed, 19 Apr 2000 19:12:11 +0200 (CEST)
Message-ID: <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de> (raw)
In-Reply-To: <00f301bfa9fd$26b21bd0$d041fcc1@bu.vtcom.fr>
> Hello
>
> I would like to know
> - how to create under psql a BLOB field in sql language
> - how store a value (and not a file) in sql language
> - the maximum size of the BLOB field
For now (and 7.0), there is no way to have rows >8K in a
regular table.
But I'm actually making alot of progress in breaking it.
Seems we'll get rid of this restriction in 7.1.
To share some info with the other developers:
I just successfully stored the entire PG sources (~10M) in
one table. Stored are all 936 .[chyl] files. The tables
schema is:
sources (
s_path text,
s_data clob
)
clob is a user defined, very simple varsize datatype, I
created for testing. The entire source tree inserts in 23
seconds. If I cut off all sources at 7K and insert into a
text field it needs 8 seconds. Not bad, because the amount
of data shrinks <4M due to the cutoff.
Reading the data back shows similar results, 0.8 secs vs. 0.3
secs.
And the toaster already uses a heap+index to store values
external.
This all done with full toasting - i.e. try to compress, and
only if it still doesn't fit move already compressed
attribute into secondary relation. As a side effect, the 10M
of sources only need 3.4M main and 2.6M secondary heap.
I experienced some trouble with occational "Deadlock"
detections, when using multiple processes hammering on the
toaster. But I was able to reproduce the same error with a
plain 7.0 and "text" attributes (only far less frequent).
Seems there's something wrong in our deadlock detection
algorithm.
Another thing is, that ISTM that index's never shrink on
vacuum. One requirement of TOAST is, that the secondaries
tables index-OID is remembered in the external reference, so
the ref can be passed around in the entire backend and
whenever needed, it's real value can be fetched quickly by an
index scan. Thus, it's not possible to drop/recreate an index
on that right now. Seems we need a "REBUILD INDEX" utility or
the like.
Anyway, TOAST is on it's way. And as soon as we go for 7.1,
I'll need alot of help to make all our existing types
toastable, add administrative utility commands and teach
pg_dump to deal with all that.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-sql@postgresql.org
Cc: wieck@debis.com, philippe.gobin@fth.net
Subject: Re: TOAST (was: BLOB)
In-Reply-To: <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de>
References: <00f301bfa9fd$26b21bd0$d041fcc1@bu.vtcom.fr> <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox