X-Original-To: pgsql-hackers-postgresql.org@postgresql.org
Received: from localhost (mx1.hub.org [200.46.208.251])
	by postgresql.org (Postfix) with ESMTP id 562309FB1B8
	for <pgsql-hackers-postgresql.org@postgresql.org>;
	Thu, 27 Jul 2006 02:15:05 -0300 (ADT)
Received: from postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024)
	with ESMTP id 38183-06 for
	<pgsql-hackers-postgresql.org@postgresql.org>;
	Thu, 27 Jul 2006 02:14:52 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-
Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
	by postgresql.org (Postfix) with ESMTP id DC78D9FB0D1
	for <pgsql-hackers@postgresql.org>;
	Thu, 27 Jul 2006 02:14:51 -0300 (ADT)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id k6R5EhxV029909;
	Thu, 27 Jul 2006 01:14:43 -0400 (EDT)
To: Mark Kirkwood <markir@paradise.net.nz>
cc: Josh Berkus <josh@agliodbs.com>, Gavin Sherry <swm@linuxworld.com.au>,
	Jie Zhang <jzhang@greenplum.com>, pgsql-hackers@postgresql.org,
	Luke Lonergan <LLonergan@greenplum.com>
Subject: Re: On-disk bitmap index patch 
In-reply-to: <44C81E32.8090500@paradise.net.nz> 
References: <C0E1CD9D.8DC0%jzhang@greenplum.com>
	<Pine.LNX.4.58.0607240720180.19004@linuxworld.com.au>
	<28525.1153698215@sss.pgh.pa.us>
	<Pine.LNX.4.58.0607240952250.19854@linuxworld.com.au>
	<28750.1153700718@sss.pgh.pa.us> <44C6652C.1020203@agliodbs.com>
	<24309.1153860389@sss.pgh.pa.us> <44C6C8CB.9030802@paradise.net.nz>
	<1593.1153923961@sss.pgh.pa.us> <44C81E32.8090500@paradise.net.nz>
Comments: In-reply-to Mark Kirkwood <markir@paradise.net.nz>
	message dated "Thu, 27 Jul 2006 14:00:18 +1200"
Date: Thu, 27 Jul 2006 01:14:43 -0400
Message-ID: <29908.1153977283@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=0.039 tagged_above=0 required=5 tests=AWL,
	SPF_HELO_PASS, SPF_PASS
X-Spam-Level: 
X-Archive-Number: 200607/1336
X-Sequence-Number: 87191

Mark Kirkwood <markir@paradise.net.nz> writes:
> An obvious deduction is that the TPCH dataset is much more amenable to 
> run compression than my synthetic Zipfian data was. The interesting 
> question is how well "real" datasets are run compressable,

Yeah --- the back-of-the-envelope calculations I was making presupposed
uniform random distribution, and we know that's often not realistic for
real datasets.  A nonuniform distribution would probably mean that some
of the bitmaps compress better-than-expected and others worse.  I have
no idea how to model that and guess what the overall result is ...

			regards, tom lane