X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id DD72B5E3632 for ; Thu, 26 Aug 2004 15:19:09 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 62328-05 for ; Thu, 26 Aug 2004 18:19:06 +0000 (GMT) Received: from candle.pha.pa.us (candle.pha.pa.us [207.106.42.251]) by svr1.postgresql.org (Postfix) with ESMTP id 7174B5E46F4 for ; Thu, 26 Aug 2004 15:18:57 -0300 (ADT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.11.6) id i7QIIrr09640; Thu, 26 Aug 2004 14:18:53 -0400 (EDT) From: Bruce Momjian Message-Id: <200408261818.i7QIIrr09640@candle.pha.pa.us> Subject: Re: Equivalent praxis to CLUSTERED INDEX? In-Reply-To: <1093544071.349.106.camel@vulture.corp.neopolitan.com> To: "J. Andrew Rogers" Date: Thu, 26 Aug 2004 14:18:53 -0400 (EDT) Cc: pgsql-performance@postgresql.org, mischa.sandberg@telus.net X-Mailer: ELM [version 2.4ME+ PL108 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200408/367 X-Sequence-Number: 8003 How do vendors actually implement auto-clustering? I assume they move rows around during quiet periods or have lots of empty space in each value bucket. --------------------------------------------------------------------------- J. Andrew Rogers wrote: > On Tue, 2004-08-24 at 22:28, Mischa Sandberg wrote: > > I see that PG has a one-shot CLUSTER command, but doesn't support > > continuously-updated clustered indexes. > > > > What I infer from newsgroup browsing is, such an index is impossible, > > given the MVCC versioning of records (happy to learn I'm wrong). > > > It is possible to have MVCC and ordered/indexed heaps, but it isn't > something you can just tack onto the currently supported types -- I > looked into this myself. It would take substantial additional code > infrastructure to support it, basically an alternative heap system and > adding support for tables with odd properties to many parts of the > system. Pretty non-trivial. > > This is probably my #1 "I wish postgres had this feature" feature. It > is a serious scalability enhancer for big systems and a pain to work > around not having. > > > > I'd be curious to know what other people, who've crossed this same > > bridge from MSSQL or Oracle or Sybase to PG, have devised, > > faced with the same kind of desired performance gain for retrieving > > blocks of rows with the same partial key. > > > The CLUSTER command is often virtually useless for precisely the kinds > of tables that need to be clustered. My databases are on-line 24x7, and > the tables that are ideal candidates for clustering are in the range of > 50-100 million rows. I can afford to lock these tables up for no more > than 5-10 minutes during off-peak in the hopes that no one notices, and > CLUSTER does not work remotely in the ballpark of that fast for tables > of that size. People who can run CLUSTER in a cron job must either have > relatively small tables or regular large maintenance windows. > > > My solution, which may or may not work for you, was to write a table > partitioning system using the natural flexibility and programmability of > postgresql (e.g. table inheritance). From this I automatically get a > roughly ordered heap according to the index I would cluster on, with > only slightly funky SQL access. The end result works much better with > CLUSTER too, though CLUSTER is much less necessary at that point > because, at least for my particular purposes, the rows are mostly > ordered due to how the data was partitioned. > > So there are ways to work around CLUSTER, but you'll have to be clever > and it will require tailoring the solution to your particular > requirements. > > > J. Andrew Rogers > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073