pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Jeffery Collins <collins@onyx-technologies.com>
Cc: pgsql-general@postgresql.org
Subject: Re: lztext and compression ratios...
Date: Wed, 05 Jul 2000 14:12:37 -0400
Message-ID: <3701.962820757@sss.pgh.pa.us> (raw)
In-Reply-To: <39636960.FEF2400C@onyx-technologies.com>
References: <39636960.FEF2400C@onyx-technologies.com>

Jeffery Collins <collins@onyx-technologies.com> writes:
> My experience with attempting to compress such a relatively small
> (around 1K) text string is that the compression ration is not very
> good.  This is because the string is not long enough for the LZ
> compression algorithm to establish really good compression patterns and
> the fact that the de-compression table has to be built into each
> record.  What I have done in the past to get around these problems is
> that I have "taught" the compression algorithm the patterns ahead of
> time and stored the de-compression patterns in an external table.  Using
> this technique, I have achieved *much* better compression ratios.

(Puts on compression-guru hat...)

There is much in what you say.  Perhaps we should consider keeping the
lztext type around (currently it's slated for doom in 7.1, since the
TOAST feature will make plain text do everything lztext does and more)
and having it be different from text in that a training sample is
supplied when the column is defined.  Not quite sure how that should
look or where to store the sample, but it could be a big win for tables
having a large number of moderate-sized text entries.

			regards, tom lane



view thread (29+ messages)  latest in thread

Message-ID: <3701.962820757@sss.pgh.pa.us>
Permalink:  ../3701.962820757@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/3701.962820757@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-sql@postgresql.org
  Cc: tgl@sss.pgh.pa.us, collins@onyx-technologies.com, pgsql-general@postgresql.org
  Subject: Re: lztext and compression ratios...
  In-Reply-To: <3701.962820757@sss.pgh.pa.us>
  References: <39636960.FEF2400C@onyx-technologies.com> <3701.962820757@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox