X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id 28DB6D1B8DF for ; Thu, 4 Mar 2004 10:06:30 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 92720-08 for ; Thu, 4 Mar 2004 06:06:13 -0400 (AST) Received: from mail.enyo.de (mail.enyo.de [212.9.189.167]) by svr1.postgresql.org (Postfix) with ESMTP id B8B95D1C9D0 for ; Thu, 4 Mar 2004 06:06:10 -0400 (AST) Received: (debugging) helo=deneb.enyo.de ip=212.9.189.171 name=deneb.enyo.de Received: from deneb.enyo.de ([212.9.189.171]) by mail.enyo.de with esmtp id 1Aypk0-0004xh-I7; Thu, 04 Mar 2004 11:06:08 +0100 Received: from fw by deneb.enyo.de with local (Exim 4.30) id 1Aypk0-0001Wu-7i; Thu, 04 Mar 2004 11:06:08 +0100 Date: Thu, 4 Mar 2004 11:06:08 +0100 To: Vivek Khera Cc: pgsql-performance@postgresql.org Subject: Re: Bulk INSERT performance in 7.4.1 Message-ID: <20040304100608.GA5521@deneb.enyo.de> References: <20040303092528.GA6647@deneb.enyo.de> <20040303141032.GA12276@deneb.enyo.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.5.1+cvs20040105i From: Florian Weimer X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200403/57 X-Sequence-Number: 5913 Vivek Khera wrote: > If you've got the time, could you try also doing the full bulk insert > test with the checkpoint log files on another physical disk? See if > that's any faster. We have been doing that for a few weeks, but the performance improvements are less than what we expected. There is hardly any disk activity on the log RAID, even during checkpointing. After I activated the tuned configuration, we are again mostly CPU-bound (it seems that updating all four indices is quite expensive). The bulk INSERT process runs single-threaded right now, and if we switched to multiple processes for that, we could reach some 1,500 INSERTs per second, I believe. This is more than sufficient for us; our real-time data collector is tuned to emit about 150 records per second, on the average. (There is an on-disk queue to compensate temporary problems, such as spikes in the data rate and database updates gone awry.) -- Current mail filters: many dial-up/DSL/cable modem hosts, and the following domains: atlas.cz, bigpond.com, freenet.de, hotmail.com, libero.it, netscape.net, postino.it, tiscali.co.uk, tiscali.cz, tiscali.it, voila.fr, wanadoo.fr, yahoo.com.