To: pgsql-performance@postgresql.org
From: Ivan Voras <ivoras@freebsd.org>
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for
	5 mins
Date: Wed, 17 Nov 2010 17:12:44 +0100
Lines: 26
Message-ID: <ic0utn$mau$1@dough.gmane.org>
References: <ic0onf$3fi$1@news.hub.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US;
	rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6
In-Reply-To: <ic0onf$3fi$1@news.hub.org>

On 11/17/10 15:26, Eric Comeau wrote:
> This is not directly a PostgreSQL performance question but I'm hoping
> some of the chaps that build high IO PostgreSQL servers on here can help.
>
> We build file transfer acceleration s/w (and use PostgreSQL as our
> database) but we need to build a test server that can handle a sustained
> write throughput of 1,25 GB for 5 mins.

Just to clarify: you need 1.25 GB/s write throughput?

For one thing, you need not only fast storage but also a fast CPU and 
file system. If you are going to stream this data directly over the 
network in a single FTP-like session, you need fast single-core 
performance (so buy the fastest low-core-count CPU possible) and a file 
system which doesn't interfere much with raw data streaming. If you're 
using Linux I'd guess either something very simple like ext2 or complex 
but designed for the task like XFS might be best. On FreeBSD, ZFS is 
great for streaming but you'll spend a lot of time tuning it :)

 From the hardware POW, since you don't really need high IOPS rates, you 
can go much cheaper with a large number of cheap desktop drives than 
with SSD-s, if you can build something like this: 
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/

You don't need the storage space here, but you *do* need many drives to 
achieve speed in RAID (remember to overdesign and assume 50 MB/s per drive).