Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes
To: "Robert Haas" <robertmhaas@gmail.com>, "Tom Lane" <tgl@sss.pgh.pa.us>,
	"Kevin Grittner" <Kevin.Grittner@wicourts.gov>
Cc: pgsql-performance@postgresql.org, "Kenneth Marshall" <ktm@rice.edu>,
	"Mladen Gogala" <mladen.gogala@vmsinfo.com>
Subject: Re: Select count(*), the sequel
References: <4CB9D89E.7050701@vmsinfo.com>
	<20101016184443.GA4993@aart.is.rice.edu>
	<AANLkTik6ifMywZq7C1_MiHLafOQ53_OXreHYQdps3GqP@mail.gmail.com>
	<2091.1288133491@sss.pgh.pa.us> <op.vk82ubgueorkce@apollo13>
	<4CC841140200002500036EA2@gw.wicourts.gov>
Date: Thu, 28 Oct 2010 11:33:16 +0200
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
From: "Pierre C" <lists@peufeu.com>
Message-ID: <op.vk94tqzceorkce@apollo13>
In-Reply-To: <4CC841140200002500036EA2@gw.wicourts.gov>
User-Agent: Opera Mail/10.62 (Linux)

> "Pierre C" <lists@peufeu.com> wrote:
>
>> in-page compression
> How would that be different from the in-page compression done by
> TOAST now?  Or are you just talking about being able to make it
> more aggressive?
> -Kevin

Well, I suppose lzo-style compression would be better used on data that is  
written a few times maximum and then mostly read (like a forum, data  
warehouse, etc). Then, good candidate pages for compression also probably  
have all tuples visible to all transactions, therefore all row headers  
would be identical and would compress very well. Of course this introduces  
a "small" problem for deletes and updates...

Delta compression is : take all the values for a column inside a page,  
look at the values and their statistical distribution, notice for example  
that they're all INTs and the values on the page fit between X+n and X-n,  
store X and only encode n with as few bits as possible for each row. This  
is only an example, the idea is to exploit the fact that on the same page,  
all the values of one column often have lots in common. xid values in row  
headers are a good example of this.

TOAST compresses datums, so it performs well on large datums ; this is the  
opposite, the idea is to compress small tuples by using the reduncancies  
between tuples.