pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Dave Cramer <pg@fastcrypt.com>
To: Greg Smith <gsmith@gregsmith.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: best use of an EMC SAN
Date: Wed, 11 Jul 2007 14:45:06 -0400
Message-ID: <DB7AF13D-BF29-436C-9E5A-12C8946D2E93@fastcrypt.com> (raw)
In-Reply-To: <Pine.GSO.4.64.0707111406250.22948@westnet.com>
References: <D02C9DBC-0127-4B62-ABE3-93646B6A3F9F@fastcrypt.com>
	<877ip7avda.fsf@oxford.xeocode.com>
	<0DA358AF-4CF3-4910-8E48-26403AE8C0AC@fastcrypt.com>
	<60zm22x2jo.fsf@dba2.int.libertyrms.com>
	<C808D1E7-6FE4-4379-B441-E46F105972AF@decibel.org>
	<Pine.GSO.4.64.0707111406250.22948@westnet.com>


On 11-Jul-07, at 2:35 PM, Greg Smith wrote:

> On Wed, 11 Jul 2007, Jim Nasby wrote:
>
>> I suppose an entirely in-memory database might be able to swamp a  
>> 2 drive WAL as well.
>
> You can really generate a whole lot of WAL volume on an EMC SAN if  
> you're doing UPDATEs fast enough on data that is mostly in-memory.   
> Takes a fairly specific type of application to do that though, and  
> whether you'll ever find it outside of a benchmark is hard to say.
>
Well, this is such an application. The db fits entirely in memory,  
and the site is doing over 12M page views a day (I'm not exactly sure  
what this translates to in transactions) .
> The main thing I would add as a consideration here is that you can  
> configure PostgreSQL to write WAL data using the O_DIRECT path,  
> bypassing the OS buffer cache, and greatly improve performance into  
> SAN-grade hardware like this.  That can be a big win if you're  
> doing writes that dirty lots of WAL, and the benefit is  
> straightforward to measure if the WAL is a dedicated section of  
> disk (just change the wal_sync_method and do benchmarks with each  
> setting).  If the WAL is just another section on an array, how well  
> those synchronous writes will mesh with the rest of the activity on  
> the system is not as straightforward to predict.  Having the WAL  
> split out provides a logical separation that makes figuring all  
> this out easier.
>
> Just to throw out a slightly different spin on the suggestions  
> going by here:  consider keeping the WAL separate, starting as a  
> RAID-1 volume, but keep 2 disks in reserve so that you could easily  
> upgrade to a 0+1 set if you end up discovering you need to double  
> the write bandwidth.  Since there's never much actual data on the  
> WAL disks that would a fairly short downtime operation.  If you  
> don't reach a wall, the extra drives might serve as spares to help  
> mitigate concerns about the WAL drives burning out faster than  
> average because of their high write volume.
>
> --
> * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com  
> Baltimore, MD
>
> ---------------------------(end of  
> broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faq




view thread (11+ messages)

Message-ID: <DB7AF13D-BF29-436C-9E5A-12C8946D2E93@fastcrypt.com>
Permalink:  ../DB7AF13D-BF29-436C-9E5A-12C8946D2E93@fastcrypt.com/
Also on:    postgresql.org/message-id/DB7AF13D-BF29-436C-9E5A-12C8946D2E93@fastcrypt.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: pg@fastcrypt.com, gsmith@gregsmith.com
  Subject: Re: best use of an EMC SAN
  In-Reply-To: <DB7AF13D-BF29-436C-9E5A-12C8946D2E93@fastcrypt.com>
  References: <D02C9DBC-0127-4B62-ABE3-93646B6A3F9F@fastcrypt.com> <877ip7avda.fsf@oxford.xeocode.com> <0DA358AF-4CF3-4910-8E48-26403AE8C0AC@fastcrypt.com> <60zm22x2jo.fsf@dba2.int.libertyrms.com> <C808D1E7-6FE4-4379-B441-E46F105972AF@decibel.org> <Pine.GSO.4.64.0707111406250.22948@westnet.com> <DB7AF13D-BF29-436C-9E5A-12C8946D2E93@fastcrypt.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox