Message-ID: <4C07DDDC.6000003@2ndquadrant.com>
Date: Thu, 03 Jun 2010 12:52:44 -0400
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Craig James <craig_james@emolecules.com>
CC: pgsql-performance@postgresql.org
Subject: Re: Weird XFS WAL problem
References: <AANLkTikS_QvlkYfv5_hbsXFH4SsTh1nP_WoTwRqpUUcz@mail.gmail.com>
	<AANLkTikegoyBvJXhqZGLZyTwRVyBs4qCuCQ7d1Vntk4w@mail.gmail.com>
	<AANLkTikjkEvj2hG-D1CMu49PQXXbxHVzpXm9JbLWzMQo@mail.gmail.com>
	<AANLkTimtdk11hrWd-OpvXGLrh-4pukGd9eHEUemEM3OR@mail.gmail.com>
	<AANLkTinAhrwwwQKh6NrX-nnvcpxT1EFFlZrUxhaDB87C@mail.gmail.com>
	<AANLkTinuOUTbuke9FTMtwSObcZBio0uOS1ysWrqFyK3t@mail.gmail.com>
	<4C06E994.2080905@emolecules.com>
In-Reply-To: <4C06E994.2080905@emolecules.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Craig James wrote:
> I'm testing/tuning a new midsize server and ran into an inexplicable 
> problem.  With an RAID10 drive, when I move the WAL to a separate 
> RAID1 drive, TPS drops from over 1200 to less than 90!

Normally <100 TPS means that the write cache on the WAL drive volume is 
disabled (or set to write-through instead of write-back).  When things 
in this area get fishy, I will usually download sysbench and have it 
specifically test how many fsync calls can happen per second.  
http://projects.2ndquadrant.com/talks , "Database Hardware 
Benchmarking", page 28 has an example of the right incantation for that.

Also, make sure you run 3ware's utilities and confirm all the disks have 
finished their initialization and verification stages.  If you just 
adjusted disk layout that and immediate launched into benchmarks, those 
are useless until the background cleanup is done.

-- 
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com   www.2ndQuadrant.us