X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id 1D1D79DCC0B for ; Tue, 7 Feb 2006 20:59:13 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 92963-02 for ; Tue, 7 Feb 2006 20:59:16 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by postgresql.org (Postfix) with ESMTP id A9EB89DCC09 for ; Tue, 7 Feb 2006 20:59:10 -0400 (AST) Received: by noel.decibel.org (Postfix, from userid 1001) id C27CD39849; Wed, 8 Feb 2006 00:59:12 +0000 (UTC) Date: Tue, 7 Feb 2006 18:59:12 -0600 From: "Jim C. Nasby" To: Simon Riggs Cc: Marc Morin , Markus Schaber , pgsql-performance@postgresql.org Subject: Re: partitioning and locking problems Message-ID: <20060208005912.GS38134@pervasive.com> References: <2BCEB9A37A4D354AA276774EE13FB8C2BCC2A6@mailserver.sandvine.com> <1139350142.1258.174.camel@localhost.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1139350142.1258.174.camel@localhost.localdomain> X-Operating-System: FreeBSD 6.0-RELEASE amd64 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.11 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.104 required=5 tests=[AWL=0.104] X-Spam-Score: 0.104 X-Spam-Level: X-Archive-Number: 200602/60 X-Sequence-Number: 17046 On Tue, Feb 07, 2006 at 10:09:02PM +0000, Simon Riggs wrote: > On Thu, 2006-02-02 at 11:27 -0500, Marc Morin wrote: > > > > > 1- long running report is running on view > > > > 2- continuous inserters into view into a table via a rule > > > > 3- truncate or rule change occurs, taking an exclusive lock. > > > > Must wait for #1 to finish. > > > > 4- new reports and inserters must now wait for #3. > > > > 5- now everyone is waiting for a single query in #1. Results > > > > in loss of insert data granularity (important for our application). > > > Using a separate lock table is what we've decided to do in this > > particular case to serialize #1 and #3. Inserters don't take this lock > > and as such will not be stalled. > > Would it not be simpler to have the Inserters change from one table to > another either upon command, on a fixed timing cycle or even better > based upon one of the inserted values (Logdate?) (or all 3?). (Requires > changes in the application layer: 3GL or db functions). Unfortunately, AFAIK rule changes would suffer from the exact same problem, which will be a serious issue for table partitioning. If you try and add a new partition while a long report is running you'll end up blocking everything. ALso, IIRC the OP was trying *not* to have the locking system impose scheduling. I believe the intention is that either 1 not block 3 or 3 not block 4. I'm honestly somewhat surprised someone hasn't run into this problem with partitioning yet; or maybe everyone who needs to do long transactions just shoves those off to slony slaves... -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461