Subject: Re: Performance PG 8.0 on dual opteron / 4GB / 3ware
From: Joost Kraaijeveld <J.Kraaijeveld@Askesis.nl>
To: Luke Lonergan <llonergan@greenplum.com>
Cc: Dave Cramer <pg@fastcrypt.com>,
	Pgsql-Performance <pgsql-performance@postgresql.org>
In-Reply-To: <BF9F6C0E.13B4A%llonergan@greenplum.com>
References: <BF9F6C0E.13B4A%llonergan@greenplum.com>
Content-Type: text/plain; charset=iso-8859-13
Content-Transfer-Encoding: quoted-printable
Date: Tue, 15 Nov 2005 20:31:56 +0100
Message-Id: <1132083116.3250.38.camel@Panoramix>
Mime-Version: 1.0

Hi Luke,

On Tue, 2005-11-15 at 10:42 -0800, Luke Lonergan wrote:
> With RAID5, it could matter a lot what block size you run your =B4dd
> bigfile=A1 test with.  You should run =B4dd if=3D/dev/zero of=3Dbigfile b=
s=3D8k
> count=3D500000=A1 for a 2GB main memory machine, multiply the count by
> (<your mem>/2GB).
If I understand correctly (I have 4GB ram):

jkr@Panoramix:~/tmp$ dd if=3D/dev/zero of=3Dbigfile bs=3D8k count=3D1000000
1000000+0 records in
1000000+0 records out
8192000000 bytes transferred in 304.085269 seconds (26939812 bytes/sec)

Which looks suspicious: 26308 MB/sec???

> It is very important with the 3Ware cards to match the driver to the
> firmware revision.
OK, I am running 1 driver behind the firmware.
  =20
>         I did notice that changing the I/O scheduler's nr_request from
>         the
>         default 128 to 1024 or even 4096 made a remarkable performance
>         improvement. I suspect that experimenting with other I/O
>         schedululers
>         could improve performance. But it is hard to find any useful
>         documentation about I/O schedulers.
>        =20
> You could try deadline, there=FFs no harm, but I=FFve found that when you
> reach the point of experimenting with schedulers, you are probably not
> addressing the real problem.
It depends. I/O Schedulers (I assume) have a purpose: some schedulers
should be more appropriate for some circumstances. And maybe my specific
circumstances (converting a database with *many updates*) is a specific
circumstance. I really don't know....

> On a 3Ware 9500 with HW RAID5 and 4 or more disks I think you should
> get 100MB/s write rate, which is double what Postgres can use.  We
> find that Postgres, even with fsync=3Dfalse, will only run at a net COPY
> speed of about 8-12 MB/s, where 12 is the Bizgres number.  8.1 might
> do 10.  But to get the 10 or 12, the WAL writing and other writing is
> about 4-5X more than the net write speed, or the speed at which the
> input file is parsed and read into the database.
As I have an (almost) seperate WAL disk: iostat does not show any
significant writing on the WAL disk....

> So, if you can get your =B4dd bigfile=A1 test to write data at 50MB/s+
> with a blocksize of 8KB, you should be doing well enough.
See above.

> Incidentally, we also find that using the XFS filesystem and setting
> the readahead to 8MB or more is extremely beneficial for performance
> with the 3Ware cards (and with others, but especially for the older
> 3Ware cards).
I don't have problems with my read performance but *only* with my
*update* performance (and not even insert performance). But than again I
am not the only one with these problems:

http://www.issociate.de/board/goto/894541/3ware_+_RAID5_
+_xfs_performance.html#msg_894541
http://lkml.org/lkml/2005/4/20/110
http://seclists.org/lists/linux-kernel/2005/Oct/1171.html

I am happy to share the tables against which I am running my checks....

--=20
Groeten,

Joost Kraaijeveld
Askesis B.V.
Molukkenstraat 14
6524NB Nijmegen
tel: 024-3888063 / 06-51855277
fax: 024-3608416
e-mail: J.Kraaijeveld@Askesis.nl
web: www.askesis.nl=20