User-Agent: Microsoft-Entourage/11.2.1.051004
Date: Tue, 31 Jan 2006 17:35:32 -0800
Subject: Re: Huge Data sets, simple queries
From: "Luke Lonergan" <llonergan@greenplum.com>
To: "Steinar H. Gunderson" <sgunderson@bigfoot.com>
cc: "Jeffrey W. Baker" <jwbaker@acm.org>,
	pgsql-performance@postgresql.org
Message-ID: <C0055064.1B67C%llonergan@greenplum.com>
Thread-Topic: [PERFORM] Huge Data sets, simple queries
Thread-Index: AcYmz8pBCOnQdpLDEdqBvAANk63kWA==
In-Reply-To: <20060201012615.GA31927@uio.no>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Steinar,

On 1/31/06 5:26 PM, "Steinar H. Gunderson" <sgunderson@bigfoot.com> wrote:

> cassarossa:~# grep md1 /proc/mdstat
> md1 : active raid1 sdf6[1] sda6[0]
> cassarossa:~# dd if=/dev/sda6 of=/dev/null bs=8k count=400000
> [system at about 35% wait for I/O and 15% system, according to top]
> 400000+0 records in
> 400000+0 records out
> 3276800000 bytes transferred in 54,488154 seconds (60137842 bytes/sec)
> [system at about 45% wait for I/O and 7% system -- whoa?]
> 400000+0 records in
> 400000+0 records out
> 3276800000 bytes transferred in 52,523771 seconds (62386990 bytes/sec)
> 
> I'm not sure if it _refutes_ the assertion that the Linux RAID-1 driver can
> do balancing of sequential reads, but it certainly doesn't present very much
> evidence in that direction. BTW, sda and sdf are on different channels of a
> dual-channel (onboard, connected via PCI-X) Adaptec board, so I doubt the bus
> is the limiting factor.

Yep - 2MB/s is noise.  Run a RAID0, you should get 120MB/s.

Incidentally, before this thread took a turn to RAID10 vs. RAID5, the
question of HW RAID adapter versus SW RAID was the focus.  I routinely see
numbers like 20MB/s coming from HW RAID adapters on Linux, so it's nice to
see someone post a decent number using SW RAID.

We're very happy with the 3Ware HW RAID adapters, but so far they're the
only ones (I have two Arecas but I mistakenly ordered PCI-E so I can't test
them :-( 

- Luke