From: Dimitri <dimitrik.fr@gmail.com>
To: Greg Smith <greg@2ndquadrant.com>
Cc: Eric Comeau <Eric.Comeau@signiant.com>
Cc: Merlin Moncure <mmoncure@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
Date: Sat, 20 Nov 2010 11:16:42 +0100
Message-ID: <AANLkTimC_VVWvg46MZBdXPtMPms0uHiUmLNPkNOhDR+F@mail.gmail.com> (raw)
In-Reply-To: <4CE468A4.80007@2ndquadrant.com>
References: <ic0onf$3fi$1@news.hub.org>
<AANLkTiky7UGdCbpdrNMAiwo69WFgv0sEah=_ov0nCN0J@mail.gmail.com>
<35FABCF85D99464FB00BC5123DC2A70A0B7EFC38@s228130hz1ew09.apptix-01.savvis.net>
<4CE468A4.80007@2ndquadrant.com>
You may also try the Sun's F5100 (flash storage array) - you may
easily get 700 MB/s just with a single I/O stream (single process), so
just with 2 streams you'll get your throughput.. - The array has 2TB
total space and max throughput should be around 4GB/s..
Rgds,
-Dimitri
On 11/18/10, Greg Smith <greg@2ndquadrant.com> wrote:
> Eric Comeau wrote:
>> Ideally 1 large file, but it may have to be multiple. We find that if
>> we send multiple files it just causes the disk to thrash more so we
>> get better throughput by sending one large file.
>
> If it's really one disk, sure. The problem you're facing is that your
> typical drive controller is going to top out at somewhere between 300 -
> 500MB/s of sequential writes before it becomes the bottleneck. Above
> somewhere between 6 and 10 drives attached to one controller on current
> hardware, adding more to a RAID-0 volume only increases the ability to
> handle seeks quickly. If you want to try and do this with traditional
> hard drives, I'd guess you'd need 3 controllers with at least 4
> short-stroked drives attached to each to have any hope of hitting
> 1.25GB/s. Once you do that, you'll run into CPU time as the next
> bottleneck. At that point, you'll probably need one CPU per controller,
> all writing out at once, to keep up with your target.
>
> The only popular hardware design that comes to mind aimed at this sort
> of thing was Sun's "Thumper" design, most recently seen in the Sun Fire
> X4540. That put 8 controllers with 6 disks attached to each, claiming
> "demonstrated up to 2 GB/sec from disk to network". It will take a
> design like that, running across multiple controllers, to get what
> you're looking for on the disk side--presuming everything else keeps up.
>
> One of the big SSD-on-PCI-e designs mentioned here already may very well
> end up being a better choice for you here though, as those aren't going
> to require quite as much hardware all get wired up.
>
> --
> Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
> PostgreSQL Training, Services and Support www.2ndQuadrant.us
> "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: dimitrik.fr@gmail.com, greg@2ndquadrant.com, Eric.Comeau@signiant.com, mmoncure@gmail.com
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
In-Reply-To: <AANLkTimC_VVWvg46MZBdXPtMPms0uHiUmLNPkNOhDR+F@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox