User-Agent: Microsoft-Entourage/11.2.5.060620
Date: Wed, 26 Jul 2006 20:55:38 -0700
Subject: Re: On-disk bitmap index patch
From: "Luke Lonergan" <llonergan@greenplum.com>
To: "Tom Lane" <tgl@sss.pgh.pa.us>,
	"Mark Kirkwood" <markir@paradise.net.nz>
cc: "Josh Berkus" <josh@agliodbs.com>, "Gavin Sherry" <swm@linuxworld.com.au>,
	"Jie Zhang" <jzhang@greenplum.com>, pgsql-hackers@postgresql.org
Message-ID: <C0ED874A.2AFBB%llonergan@greenplum.com>
Thread-Topic: [HACKERS] On-disk bitmap index patch
Thread-Index: AcaxMIVTw+bUpB0jEdupjgAWy4o9DA==
In-Reply-To: <1593.1153923961@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom,

On 7/26/06 7:26 AM, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> I wonder
> whether they oughtn't use 16-bit instead of 8-bit HRL_WORDs

We tried variations from 8-bit to 32-bit and came to the conclusion that
8-bit was a better match for the more general case.  For the moment I forget
exactly why (maybe Jie or Ayush will recall).  It's a #define I believe, so
you can play with it to find out.

There's a lot more to be done with this - I think we've got some big gains
ahead.

BTW - lots of excitement here at OSCON about bitmap index, and there's a
fellow who is using the current Bizgres version, but wants *higher*
cardinality support, so I discussed Jie's thoughts about implementing
binning on values, basically combining bit vectors into value bins as the
cardinality increases.

I am still puzzled why our index creation time increases so dramatically as
we approach cardinality 10,000.  I know that we found that index page
traversal for append began to take a lot of time as we started to increase
the number of bitvector pages - we had talked about aggregating the append
operations into groups before they were implemented to sequentialize the
access.

- Luke