Received: from localhost (maia-2.hub.org [200.46.204.187]) by postgresql.org (Postfix) with ESMTP id F2C619FBA35 for ; Mon, 21 May 2007 02:18:05 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.187]) (amavisd-maia, port 10024) with ESMTP id 74602-10 for ; Mon, 21 May 2007 02:18:01 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from telogis.co.nz (telogis.co.nz [203.98.10.169]) by postgresql.org (Postfix) with ESMTP id 1D1CC9FB67D for ; Mon, 21 May 2007 02:18:01 -0300 (ADT) Received: from RALPH2 ([::ffff:192.168.3.157]) by telogis.co.nz with esmtp; Mon, 21 May 2007 17:17:59 +1200 id 00836B5A.46512B87.00003770 From: "Ralph Mason" To: "'Tom Lane'" Cc: "'Richard Huxton'" , pgsql-performance@postgresql.org References: <011b01c798d5$141365a0$3c3a30e0$@mason@telogis.com> <464D6DE1.6040001@archonet.com> <005b01c79b1f$20aeb340$620c19c0$@mason@telogis.com> <17450.1179698150@sss.pgh.pa.us> In-Reply-To: <17450.1179698150@sss.pgh.pa.us> Subject: Re: Ever Increasing IOWAIT Date: Mon, 21 May 2007 17:17:42 +1200 Message-ID: <00d501c79b67$5bf18480$13d48d80$@mason@telogis.com> MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1250" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcebKfWcYDTranBwRoWZc6Yf8W7ZKAAPHhbQ Content-Language: en-nz X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.276 tagged_above=0 required=5 tests=AWL=-0.528, BAYES_05=-1.11, MSGID_MULTIPLE_AT=1.914 X-Spam-Level: X-Archive-Number: 200705/388 X-Sequence-Number: 24778 "Ralph Mason" writes: > Ralph Mason wrote: >> We have a database running on a 4 processor machine. As time goes by >> the IO gets worse and worse peeking at about 200% as the machine loads up. >> >> The weird thing is that if we restart postgres it's fine for hours but >> over time it goes bad again. >Do you by any chance have stats collection enabled and >stats_reset_on_server_start set to true? If so, maybe this is explained >by growth in the size of the stats file over time. It'd be interesting >to keep an eye on the size of $PGDATA/global/pgstat.stat over a fast-to- >slow cycle. We do because we use the stats to figure out when we will vacuum. Our vacuum process reads that table and when it runs resets it using pg_stat_reset() to clear it down each time it runs (about ever 60 seconds when the db is very busy), stats_reset_on_server_restart is off. Interestingly after a suggestion here I went and looked at the IO stat at the same time. It shows the writes as expected and picking up exactly where they were before the reset, but the reads drop dramatically - like it's reading far less data after the reset. I will watch the size of the pgstat.stat table. Ralph -- Internal Virus Database is out-of-date. Checked by AVG Free Edition. Version: 7.5.432 / Virus Database: 268.15.9/573 - Release Date: 5/12/2006 4:07 p.m.