Date: Wed, 13 Dec 2017 13:55:01 -0300
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Custom compression methods
Message-ID: <20171213165501.rita5g7kepdcimwr@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b1e047de-67d5-2fd8-ad9e-93434497ad91@2ndquadrant.com>
User-Agent: NeoMutt/20170306-137-4415bd-dirty (1.8.0)
Precedence: bulk

Tomas Vondra wrote:

> On 12/13/2017 01:54 AM, Robert Haas wrote:

> > 3. Compression is only applied to large-ish values.  If you are just
> > making the data type representation more compact, you probably want to
> > apply the new representation to all values.  If you are compressing in
> > the sense that the original data gets smaller but harder to interpret,
> > then you probably only want to apply the technique where the value is
> > already pretty wide, and maybe respect the user's configured storage
> > attributes.  TOAST knows about some of that kind of stuff.
> 
> Good point. One such parameter that I really miss is compression level.
> I can imagine tuning it through CREATE COMPRESSION METHOD, but it does
> not seem quite possible with compression happening in a datatype.

Hmm, actually isn't that the sort of thing that you would tweak using a
column-level option instead of a compression method?
  ALTER TABLE ALTER COLUMN SET (compression_level=123)
The only thing we need for this is to make tuptoaster.c aware of the
need to check for a parameter.

> > I don't think TOAST needs to be entirely transparent for the
> > datatypes.  We've already dipped our toe in the water by allowing some
> > operations on "short" varlenas, and there's really nothing to prevent
> > a given datatype from going further.  The OID problem you mentioned
> > would presumably be solved by hard-coding the OIDs for any built-in,
> > privileged compression methods.
> 
> Stupid question, but what do you mean by "short" varlenas?

Those are varlenas with 1-byte header rather than the standard 4-byte
header.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services