Received: from localhost (unknown [200.46.204.183])
	by developer.postgresql.org (Postfix) with ESMTP id 076912E004A
	for <pgsql-performance-postgresql.org@developer.postgresql.org>;
	Tue, 13 May 2008 17:09:34 -0300 (ADT)
Received: from developer.postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024)
	with ESMTP id 31127-05-2
	for <pgsql-performance-postgresql.org@developer.postgresql.org>;
	Tue, 13 May 2008 17:09:23 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from pih-relay04.plus.net (pih-relay04.plus.net [212.159.14.131])
	by developer.postgresql.org (Postfix) with ESMTP id 910102E008B
	for <pgsql-performance@postgresql.org>;
	Tue, 13 May 2008 17:08:20 -0300 (ADT)
Received: from [80.229.150.39] (helo=mansionfamily.plus.com)
	by pih-relay04.plus.net with esmtp (Exim) id 1Jw0nI-0005PA-KJ
	for pgsql-performance@postgresql.org; Tue, 13 May 2008 21:08:16 +0100
Received: from [192.168.0.87] ([192.168.0.87]:2208)
	by mansionfamily.plus.com with [XMail 1.22 ESMTP Server]
	id <S195A5> for <pgsql-performance@postgresql.org> from
	<james@mansionfamily.plus.com>; Tue, 13 May 2008 21:14:12 +0100
Message-ID: <4829F64A.9090109@mansionfamily.plus.com>
Date: Tue, 13 May 2008 21:12:58 +0100
From: James Mansion <james@mansionfamily.plus.com>
Organization: MsgWare
User-Agent: Thunderbird 2.0.0.14 (Windows/20080421)
MIME-Version: 1.0
To: PFC <lists@peufeu.com>
CC: Francisco Reyes <lists@stringsutils.com>,
	Pgsql performance <pgsql-performance@postgresql.org>
Subject: Re: RAID controllers for Postgresql on large setups
References: <cone.1210644243.416947.84940.1000@zoraida.natserv.net>
	<op.ua26u3iacigqcu@apollo13.peufeu.com>
In-Reply-To: <op.ua26u3iacigqcu@apollo13.peufeu.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Plusnet-Relay: 314879222711305e53bd5898190ad0fb
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200805/173
X-Sequence-Number: 30141

PFC wrote:
>     PCI limits you to 133 MB/s (theoretical), actual speed being 
> around 100-110 MB/s.
Many servers do have more than one bus.  You have to process that data 
too so its not going to be as much of a limit as you are suggesting.  It 
may be possible to stream a compressed data file to the server and copy 
in from that after decompression, which will free LAN bandwidth.  Or 
even if you RPC blocks of compressed data and decompress in the proc and 
insert right there.

>     On your current setup with 15K drives if you need 1 fsync per 
> INSERT you won't do more than 250 per second, which is very limiting... 
Well, thats 250 physical syncs.  But if you have multiple insert streams 
(for group commit), or can batch the rows in each insert or copy, its 
not necessarily as much of a problem as you seem to be implying. 
Particularly if you are doing the holding table trick.

James