X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 3A19F329CF7 for ; Wed, 8 Sep 2004 17:06:05 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 34140-04 for ; Wed, 8 Sep 2004 16:05:51 +0000 (GMT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by svr1.postgresql.org (Postfix) with ESMTP id 0C111329CB7 for ; Wed, 8 Sep 2004 17:05:35 +0100 (BST) Received: from news.hub.org (news.hub.org [200.46.204.72]) by news.hub.org (8.12.9/8.12.9) with ESMTP id i88G5YCn035941 for ; Wed, 8 Sep 2004 16:05:34 GMT (envelope-from news@news.hub.org) Received: (from news@localhost) by news.hub.org (8.12.9/8.12.9/Submit) id i88FcqGd026260 for pgsql-performance@postgresql.org; Wed, 8 Sep 2004 15:38:52 GMT From: Mischa Sandberg Reply-To: ischamay.andbergsay@activestateway.com User-Agent: Mozilla Thunderbird 0.7.3 (X11/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 X-Newsgroups: comp.databases.postgresql.performance Subject: Re: Equivalent praxis to CLUSTERED INDEX? References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Lines: 36 Message-ID: Date: Wed, 08 Sep 2004 15:38:51 GMT To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=1.1 tagged_above=0.0 required=5.0 tests=NO_DNS_FOR_FROM X-Spam-Level: * X-Archive-Number: 200409/71 X-Sequence-Number: 8199 Mischa Sandberg wrote: > Coming from the MSSQL world, I'm used to the first step in optimization > to be, choose your clustered index and choose it well. > I see that PG has a one-shot CLUSTER command, but doesn't support > continuously-updated clustered indexes. > What I infer from newsgroup browsing is, such an index is impossible, > given the MVCC versioning of records (happy to learn I'm wrong). > I'd be curious to know what other people, who've crossed this same > bridge from MSSQL or Oracle or Sybase to PG, have devised, > faced with the same kind of desired performance gain for retrieving > blocks of rows with the same partial key. Just to let people know, after trying various options, this looks the most promising: - segment the original table into four tables (call them A,B,C,D) - all insertions go into A. - longterm data lives in B. - primary keys of all requests to delete rows from (B) go into D -- no actual deletions are done against B. Deletions against A happen as normal. - all queries are made against a view: a union of A and B and (not exists) D. - daily merge A,B and (where not exists...) D, into C - run cluster on C, then swap names on B and C, truncate A and D. Not rocket science, but it seems to give the payback of normal clustering without locking the table for long periods of time. It also saves on VACUUM FULL time. At present, we're only at 1M rows in B on this. More when I know it. Advance warning on any gotchas with this approach would be much appreciated. Making a complete copy of (B) is a bit of an ouch.