pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: wieck@debis.com
To: Philippe Gobin <philippe.gobin@fth.net>
Cc: pgsql-sql@postgresql.org
Subject: TOAST (was: BLOB)
Date: Wed, 19 Apr 2000 19:12:11 +0200 (CEST)
Message-ID: <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de> (raw)
In-Reply-To: <00f301bfa9fd$26b21bd0$d041fcc1@bu.vtcom.fr>

> Hello
>
> I would like to know
> - how to create under psql a BLOB field in sql language
> - how store a value (and not a file) in sql language
> - the maximum size of the BLOB field

    For  now  (and  7.0),  there  is no way to have rows >8K in a
    regular table.

    But I'm actually making alot  of  progress  in  breaking  it.
    Seems we'll get rid of this restriction in 7.1.

    To share some info with the other developers:

    I  just  successfully  stored the entire PG sources (~10M) in
    one table.  Stored are all 936  .[chyl]  files.   The  tables
    schema is:

        sources (
          s_path    text,
          s_data    clob
        )

    clob  is  a  user  defined,  very  simple varsize datatype, I
    created for testing. The entire source  tree  inserts  in  23
    seconds.  If  I  cut  off all sources at 7K and insert into a
    text field it needs 8 seconds.  Not bad, because  the  amount
    of data shrinks <4M due to the cutoff.

    Reading the data back shows similar results, 0.8 secs vs. 0.3
    secs.

    And the toaster already uses a  heap+index  to  store  values
    external.

    This  all done with full toasting - i.e. try to compress, and
    only  if  it  still  doesn't  fit  move  already   compressed
    attribute  into secondary relation. As a side effect, the 10M
    of sources only need 3.4M main and 2.6M secondary heap.

    I  experienced  some  trouble  with   occational   "Deadlock"
    detections,  when  using  multiple processes hammering on the
    toaster. But I was able to reproduce the same  error  with  a
    plain  7.0  and  "text"  attributes (only far less frequent).
    Seems there's  something  wrong  in  our  deadlock  detection
    algorithm.

    Another  thing  is,  that  ISTM  that index's never shrink on
    vacuum. One requirement of TOAST  is,  that  the  secondaries
    tables  index-OID is remembered in the external reference, so
    the ref can be  passed  around  in  the  entire  backend  and
    whenever needed, it's real value can be fetched quickly by an
    index scan. Thus, it's not possible to drop/recreate an index
    on that right now. Seems we need a "REBUILD INDEX" utility or
    the like.

    Anyway, TOAST is on it's way. And as soon as we go  for  7.1,
    I'll  need  alot  of  help  to  make  all  our existing types
    toastable, add  administrative  utility  commands  and  teach
    pg_dump to deal with all that.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#========================================= wieck@debis.com (Jan Wieck) #





view thread (13+ messages)  latest in thread

Message-ID: <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de>
Permalink:  ../m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de/
Also on:    postgresql.org/message-id/m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-sql@postgresql.org
  Cc: wieck@debis.com, philippe.gobin@fth.net
  Subject: Re: TOAST (was: BLOB)
  In-Reply-To: <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de>
  References: <00f301bfa9fd$26b21bd0$d041fcc1@bu.vtcom.fr> <m12hy1L-0003knC@orion.SAPserv.Hamburg.dsh.de>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox