User-Agent: Microsoft-Entourage/11.1.0.040913
Date: Thu, 28 Jul 2005 21:03:20 -0700
Subject: Re: [PATCHES] COPY FROM performance improvements
From: "Luke Lonergan" <llonergan@greenplum.com>
To: "Joshua D. Drake" <jd@commandprompt.com>,
	"Mark Wong" <markw@osdl.org>
Cc: "Patrick Welche" <prlw1@newn.cam.ac.uk>,
	pgsql-performance@postgresql.org, maryedie@osdl.org
Message-ID: <BF0EF898.A360%llonergan@greenplum.com>
In-Reply-To: <42E970E4.4010902@commandprompt.com>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

>>> Can you post the "time dd if=/dev/zero of=bigfile bs=8k count=500000"
>>> results?  Also do the reverse (read the file) with "time dd if=bigfile
>>> of=/dev/null bs=8k".
> 
> I didn't see this come across before... here ya go:
> 
> time dd if=/dev/zero of=bigfile bs=8k count=500000
> 
> 500000+0 records in
> 500000+0 records out
> 
> real    1m52.738s
> user    0m0.310s
> sys     0m36.590s

So, that's 35MB/s, or 1/2 of a single disk drive.
 
> time dd if=bigfile of=/dev/null bs=8k
> 
> time dd if=bigfile of=/dev/null bs=8k
> 500000+0 records in
> 500000+0 records out
> 
> real    4m38.742s
> user    0m0.320s
> sys     0m27.870s

And that's 14MB/s, or < 1/4 of a single disk drive.

> FYI on your hardware raid comment... I easily get 50 megs a second on my
> 3ware controllers and faster on my LSI SATA controllers.

Then you are almost getting one disk worth of bandwidth.

By comparison, we get this using Linux software RAID on Xeon or Opteron:

$ time dd if=/dev/zero of=bigfile bs=8k count=500000
500000+0 records in
500000+0 records out

real    0m26.927s
user    0m0.074s
sys     0m8.769s

$ time dd if=bigfile of=/dev/null bs=8k
500000+0 records in
500000+0 records out

real    0m28.190s
user    0m0.039s
sys     0m8.349s

with less CPU usage than HW SCSI RAID controllers.

- Luke