X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id 077C79DCB62 for ; Wed, 22 Mar 2006 08:55:16 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 35428-10 for ; Wed, 22 Mar 2006 08:55:19 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by postgresql.org (Postfix) with ESMTP id CC31E9DCB34 for ; Wed, 22 Mar 2006 08:55:14 -0400 (AST) Received: by noel.decibel.org (Postfix, from userid 1001) id 6A6D656435; Wed, 22 Mar 2006 06:55:18 -0600 (CST) Received: (hashcash-sendmail, from uid 1001); Wed, 22 Mar 2006 06:55:17 -0600 Date: Wed, 22 Mar 2006 06:55:17 -0600 From: "Jim C. Nasby" To: Mikael Carneholm Cc: Simon Riggs , pgsql-performance@postgresql.org Subject: Re: Migration study, step 1: bulk write performanceoptimization Message-ID: <20060322125516.GB15742@pervasive.com> References: <7F10D26ECFA1FB458B89C5B4B0D72C2B08828E@sesrv12.wirelesscar.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7F10D26ECFA1FB458B89C5B4B0D72C2B08828E@sesrv12.wirelesscar.com> X-Operating-System: FreeBSD 6.0-RELEASE-p4 amd64 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.11 X-Hashcash: 1:20:060322:mikael.carneholm@wirelesscar.com::6h3FNmQnWmpQiQd5:00000 0000000000000000000000005KXB X-Hashcash: 1:20:060322:simon@2ndquadrant.com::JtRTKGL8hFb9dwmm:0000000000000000 0000000000000000000000004gTJ X-Hashcash: 1:20:060322:pgsql-performance@postgresql.org::qgiaPmO7tktkYm04:00000 00000000000000000000000053R3 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.107 required=5 tests=[AWL=0.107] X-Spam-Score: 0.107 X-Spam-Level: X-Archive-Number: 200603/448 X-Sequence-Number: 17885 On Wed, Mar 22, 2006 at 10:04:49AM +0100, Mikael Carneholm wrote: > It does ("LOG: checkpoints are occurring too frequently (2 seconds apart)") However, I tried increasing checkpoint_segments to 32 (512Mb) making it checkpoint every 15 second or so, but that gave a more uneven insert rate than with checkpoint_segments=3. Maybe 64 segments (1024Mb) would be a better value? If I set checkpoint_segments to 64, what would a reasonable bgwriter setup be? I still need to improve my understanding of the relations between checkpoint_segments <-> shared_buffers <-> bgwriter... :/ Probably the easiest way is to set checkpoint_segments to something like 128 or 256 (or possibly higher), and then make bg_writer more aggressive by increasing bgwriter_*_maxpages dramatically (maybe start with 200). You might want to up lru_percent as well, otherwise it will take a minimum of 20 seconds to fully scan. Basically, slowly start increasing settings until performance smooths out. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461