X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id 02739D1D5B0 for ; Fri, 13 Feb 2004 06:29:35 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 55725-07 for ; Fri, 13 Feb 2004 02:29:34 -0400 (AST) Received: from davinci.ethosmedia.com (server228.ethosmedia.com [209.128.84.228]) by svr1.postgresql.org (Postfix) with ESMTP id 3660BD1D597 for ; Fri, 13 Feb 2004 02:29:33 -0400 (AST) Received: from [63.195.55.98] (HELO 192.168.1.29) by davinci.ethosmedia.com (CommuniGate Pro SMTP 4.0.2) with ESMTP id 4419577; Thu, 12 Feb 2004 22:30:37 -0800 From: Josh Berkus Reply-To: josh@agliodbs.com Organization: Aglio Database Solutions To: "Marinos J. Yannikos" , pgsql-performance@postgresql.org Subject: Re: optimization ideas for frequent, large(ish) updates in frequently accessed DB? Date: Thu, 12 Feb 2004 22:28:41 -0800 User-Agent: KMail/1.5.4 References: <402C213A.4030105@geizhals.at> In-Reply-To: <402C213A.4030105@geizhals.at> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200402122228.41680.josh@agliodbs.com> X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200402/153 X-Sequence-Number: 5677 Marinos, > shared_buffers=100000 > (I tried many values, this seems to work well for us - 12GB RAM) > wal_buffers=500 > sort_mem=800000 > checkpoint_segments=16 > effective_cache_size=1000000 > etc. 800MB for sort mem? Are you sure you typed that correctly? You must be counting on not having a lot of concurrent queries. It sure will speed up index updating, though! I think you might do well to experiment with using the checkpoint_delay and checkpoint_sibilings settings in order to get more efficient batch processing of updates while selects are going on. I would also suggest increasing checkpoint segments as much as your disk space will allow; I know one reporting database I run that does batch loads is using 128 (which is about a gig of disk, I think). What have you set max_fsm_relations and max_fsm_pages to? The latter should be very high for you, like 10,000,000 For that matter, what *version* of PostgreSQL are you running? Also, make sure that your tables get vaccuumed regularly. > Any help/suggestions would be greatly appreciated... Even if it's > something like "you need a faster db box, there's no other way" ;-) Well, a battery-backed RAID controller with a fast cache would certainly help. You'll also be glad to know that a *lot* of the improvements in the upcoming PostgreSQL 7.5 are aimed at giving better peformance on large, high-activity databases like yours. -- -Josh Berkus Aglio Database Solutions San Francisco