pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Quad processor options
25+ messages / 14 participants
[nested] [flat]

* Quad processor options
@ 2004-05-11 19:06 Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:14 ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 22:23 ` Re: Quad processor options J. Andrew Rogers <jrogers@neopolitan.com>
  2004-05-12 04:03 ` Re: [PERFORM] Quad processor options Dennis Bjorklund <db@zigo.dhs.org>
  0 siblings, 4 replies; 25+ messages in thread

From: Bjoern Metzdorf @ 2004-05-11 19:06 UTC (permalink / raw)
  To: pgsql-performance; +Cc: Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

Hi,

I am curious if there are any real life production quad processor setups 
running postgresql out there. Since postgresql lacks a proper 
replication/cluster solution, we have to buy a bigger machine.

Right now we are running on a dual 2.4 Xeon, 3 GB Ram and U160 SCSI 
hardware-raid 10.

Has anyone experiences with quad Xeon or quad Opteron setups? I am 
looking at the appropriate boards from Tyan, which would be the only 
option for us to buy such a beast. The 30k+ setups from Dell etc. don't 
fit our budget.

I am thinking of the following:

Quad processor (xeon or opteron)
5 x SCSI 15K RPM for Raid 10 + spare drive
2 x IDE for system
ICP-Vortex battery backed U320 Hardware Raid
4-8 GB Ram

Would be nice to hear from you.

Regards,
Bjoern



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 20:14 ` Paul Tuckfield <paul@tuckfield.com>
  2004-05-11 20:46   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  3 siblings, 1 reply; 25+ messages in thread

From: Paul Tuckfield @ 2004-05-11 20:14 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>; pgsql-performance

it's very good to understand specific choke points you're trying to 
address by upgrading so you dont get disappointed.  Are you truly CPU 
constrained, or is it memory footprint or IO thruput that makes you 
want to upgrade?

IMO The best way to begin understanding system choke points is vmstat 
output.

Would you mind forwarding the output of "vmstat 10 120" under peak load 
period?  (I'm asusming this is linux or unix variant)  a brief 
description of what is happening during the vmstat sample would help a 
lot too.



> I am curious if there are any real life production quad processor 
> setups running postgresql out there. Since postgresql lacks a proper 
> replication/cluster solution, we have to buy a bigger machine.
>
> Right now we are running on a dual 2.4 Xeon, 3 GB Ram and U160 SCSI 
> hardware-raid 10.
>
> Has anyone experiences with quad Xeon or quad Opteron setups? I am 
> looking at the appropriate boards from Tyan, which would be the only 
> option for us to buy such a beast. The 30k+ setups from Dell etc. 
> don't fit our budget.
>
> I am thinking of the following:
>
> Quad processor (xeon or opteron)
> 5 x SCSI 15K RPM for Raid 10 + spare drive
> 2 x IDE for system
> ICP-Vortex battery backed U320 Hardware Raid
> 4-8 GB Ram
>
> Would be nice to hear from you.
>
> Regards,
> Bjoern
>
> ---------------------------(end of 
> broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:14 ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
@ 2004-05-11 20:46   ` Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-12 05:34     ` Re: Quad processor options spied@yandex.ru
  0 siblings, 1 reply; 25+ messages in thread

From: Bjoern Metzdorf @ 2004-05-11 20:46 UTC (permalink / raw)
  To: Paul Tuckfield <paul@tuckfield.com>; +Cc: Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>; pgsql-performance

Paul Tuckfield wrote:

> Would you mind forwarding the output of "vmstat 10 120" under peak load 
> period?  (I'm asusming this is linux or unix variant)  a brief 
> description of what is happening during the vmstat sample would help a 
> lot too.

see my other mail.

We are running Linux, Kernel 2.4. As soon as the next debian version 
comes out, I'll happily switch to 2.6 :)

Regards,
Bjoern



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:14 ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
  2004-05-11 20:46   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-12 05:34     ` spied@yandex.ru
  0 siblings, 0 replies; 25+ messages in thread

From: spied@yandex.ru @ 2004-05-12 05:34 UTC (permalink / raw)
  To: pgsql-performance

BM> see my other mail.

BM> We are running Linux, Kernel 2.4. As soon as the next debian version 
BM> comes out, I'll happily switch to 2.6 :)

it's very simple to use 2.6 with testing version, but if you like
woody - you can simple install several packets from testing or
backports.org

if you think about perfomance you must use lastest version of
postgresql server - it can be installed from testing or backports.org
too (but postgresql from testing depend from many other testing
packages).

i think if you upgade existing system you can use backports.org for
nevest packages, if you install new - use testing - it can be used
on production servers today




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 20:16 ` scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  3 siblings, 1 reply; 25+ messages in thread

From: scott.marlowe @ 2004-05-11 20:16 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: pgsql-performance; Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

On Tue, 11 May 2004, Bjoern Metzdorf wrote:

> Hi,
> 
> I am curious if there are any real life production quad processor setups 
> running postgresql out there. Since postgresql lacks a proper 
> replication/cluster solution, we have to buy a bigger machine.
> 
> Right now we are running on a dual 2.4 Xeon, 3 GB Ram and U160 SCSI 
> hardware-raid 10.
> 
> Has anyone experiences with quad Xeon or quad Opteron setups? I am 
> looking at the appropriate boards from Tyan, which would be the only 
> option for us to buy such a beast. The 30k+ setups from Dell etc. don't 
> fit our budget.
> 
> I am thinking of the following:
> 
> Quad processor (xeon or opteron)
> 5 x SCSI 15K RPM for Raid 10 + spare drive
> 2 x IDE for system
> ICP-Vortex battery backed U320 Hardware Raid
> 4-8 GB Ram

Well, from what I've read elsewhere on the internet, it would seem the 
Opterons scale better to 4 CPUs than the basic Xeons do.  Of course, the 
exception to this is SGI's altix, which uses their own chipset and runs 
the itanium with very good memory bandwidth.

But, do you really need more CPU horsepower?

Are you I/O or CPU or memory or memory bandwidth bound?  If you're sitting 
at 99% idle, and iostat says your drives are only running at some small 
percentage of what you know they could, you might be memory or memory 
bandwidth limited.  Adding two more CPUs will not help with that 
situation.

If your I/O is saturated, then the answer may well be a better RAID 
array, with many more drives plugged into it.  Do you have any spare 
drives you can toss on the machine to see if that helps?  Sometimes going 
from 4 drives in a RAID 1+0 to 6 or 8 or more can give a big boost in 
performance.

In short, don't expect 4 CPUs to solve the problem if the problem isn't 
really the CPUs being maxed out.

Also, what type of load are you running?  Mostly read, mostly written, few 
connections handling lots of data, lots of connections each handling a 
little data, lots of transactions, etc...

If you are doing lots of writing, make SURE you have a controller that 
supports battery backed cache and is configured to write-back, not 
write-through.




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
@ 2004-05-11 20:41   ` Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 22:46     ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
  0 siblings, 2 replies; 25+ messages in thread

From: Bjoern Metzdorf @ 2004-05-11 20:41 UTC (permalink / raw)
  To: scott.marlowe <scott.marlowe@ihs.com>; +Cc: pgsql-performance@postgresql.org, "Pgsql-Admin (E-mail)" <pgsql-admin@postgresql.org>

scott.marlowe wrote:

> Well, from what I've read elsewhere on the internet, it would seem the 
> Opterons scale better to 4 CPUs than the basic Xeons do.  Of course, the 
> exception to this is SGI's altix, which uses their own chipset and runs 
> the itanium with very good memory bandwidth.

This is basically what I read too. But I cannot spent money on a quad 
opteron just for testing purposes :)

> But, do you really need more CPU horsepower?
> 
> Are you I/O or CPU or memory or memory bandwidth bound?  If you're sitting 
> at 99% idle, and iostat says your drives are only running at some small 
> percentage of what you know they could, you might be memory or memory 
> bandwidth limited.  Adding two more CPUs will not help with that 
> situation.

Right now we have a dual xeon 2.4, 3 GB Ram, Mylex extremeraid 
controller, running 2 Compaq BD018122C0, 1 Seagate ST318203LC and 1 
Quantum ATLAS_V_18_SCA.

iostat show between 20 and 60 % user avg-cpu. And this is not even peak 
time.

I attached a "vmstat 10 120" output for perhaps 60-70% peak load.

> If your I/O is saturated, then the answer may well be a better RAID 
> array, with many more drives plugged into it.  Do you have any spare 
> drives you can toss on the machine to see if that helps?  Sometimes going 
> from 4 drives in a RAID 1+0 to 6 or 8 or more can give a big boost in 
> performance.

Next drives I'll buy will certainly be 15k scsi drives.

> In short, don't expect 4 CPUs to solve the problem if the problem isn't 
> really the CPUs being maxed out.
> 
> Also, what type of load are you running?  Mostly read, mostly written, few 
> connections handling lots of data, lots of connections each handling a 
> little data, lots of transactions, etc...

In peak times we can get up to 700-800 connections at the same time. 
There are quite some updates involved, without having exact numbers I'll 
think that we have about 70% selects and 30% updates/inserts.

> If you are doing lots of writing, make SURE you have a controller that 
> supports battery backed cache and is configured to write-back, not 
> write-through.

Could you recommend a certain controller type? The only battery backed 
one that I found on the net is the newest model from icp-vortex.com.

Regards,
Bjoern

Attachments:

vmstat.txt (text/plain, 1.3K) download ~# vmstat 10 120 procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 1 0 24180 10584 32468 2332208 0 1 0 2 1 2 2 0 0 0 2 0 24564 10480 27812 2313528 8 0 7506 574 1199 8674 30 7 63 2 1 0 24692 10060 23636 2259176 0 18 8099 298 2074 6328 25 7 68 2 0 0 24584 18576 21056 2299804 3 6 13208 305 1598 8700 23 6 71 1 21 1 24504 16588 20912 2309468 4 0 1442 1107 754 6874 42 13 45 6 1 0 24632 13148 19992 2319400 0 0 2627 499 1184 9633 37 6 58 5 1 0 24488 10912 19292 2330080 5 0 3404 150 1466 10206 32 6 61 4 1 0 24488 12180 18824 2342280 3 0 2934 40 1052 3866 19 3 78 0 0 0 24420 14776 19412 2347232 6 0 403 216 1123 4702 22 3 74 0 0 0 24548 14408 17380 2321780 4 0 522 715 965 6336 25 5 71 4 0 0 24676 12504 17756 2322988 0 0 564 830 883 7066 31 6 63 0 3 0 24676 14060 18232 2325224 0 0 483 388 1097 3401 21 3 76 0 2 1 24676 13044 18700 2322948 0 0 701 195 1078 5187 23 3 74 2 0 0 24676 21576 18752 2328168 0 0 467 177 1552 3574 18 3 78
^ permalink raw reply [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 21:02     ` scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 21:11       ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-12 04:58       ` Re: [PERFORM] Quad processor options Grega Bremec <grega.bremec@noviforum.si>
  1 sibling, 2 replies; 25+ messages in thread

From: scott.marlowe @ 2004-05-11 21:02 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: pgsql-performance; Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

On Tue, 11 May 2004, Bjoern Metzdorf wrote:

> scott.marlowe wrote:
> 
> > Well, from what I've read elsewhere on the internet, it would seem the 
> > Opterons scale better to 4 CPUs than the basic Xeons do.  Of course, the 
> > exception to this is SGI's altix, which uses their own chipset and runs 
> > the itanium with very good memory bandwidth.
> 
> This is basically what I read too. But I cannot spent money on a quad 
> opteron just for testing purposes :)

Wouldn't it be nice to just have a lab full of these things?

> > If your I/O is saturated, then the answer may well be a better RAID 
> > array, with many more drives plugged into it.  Do you have any spare 
> > drives you can toss on the machine to see if that helps?  Sometimes going 
> > from 4 drives in a RAID 1+0 to 6 or 8 or more can give a big boost in 
> > performance.
> 
> Next drives I'll buy will certainly be 15k scsi drives.

Better to buy more 10k drives than fewer 15k drives.  Other than slightly 
faster select times, the 15ks aren't really any faster.

> > In short, don't expect 4 CPUs to solve the problem if the problem isn't 
> > really the CPUs being maxed out.
> > 
> > Also, what type of load are you running?  Mostly read, mostly written, few 
> > connections handling lots of data, lots of connections each handling a 
> > little data, lots of transactions, etc...
> 
> In peak times we can get up to 700-800 connections at the same time. 
> There are quite some updates involved, without having exact numbers I'll 
> think that we have about 70% selects and 30% updates/inserts.

Wow, a lot of writes then.

> > If you are doing lots of writing, make SURE you have a controller that 
> > supports battery backed cache and is configured to write-back, not 
> > write-through.
> 
> Could you recommend a certain controller type? The only battery backed 
> one that I found on the net is the newest model from icp-vortex.com.

Sure, adaptec makes one, so does lsi megaraid.  Dell resells both of 
these, the PERC3DI and the PERC3DC are adaptec, then lsi in that order, I 
believe.  We run the lsi megaraid with 64 megs battery backed cache.

Intel also makes one, but I've heard nothing about it.

If you get the LSI megaraid, make sure you're running the latest megaraid 
2 driver, not the older, slower 1.18 series.  If you are running linux, 
look for the dkms packaged version.  dkms, (Dynamic Kernel Module System) 
automagically compiles and installs source rpms for drivers when you 
install them, and configures the machine to use them to boot up.  Most 
drivers seem to be slowly headed that way in the linux universe, and I 
really like the simplicity and power of dkms.

I haven't directly tested anything but the adaptec and the lsi megaraid.  
Here at work we've had massive issues trying to get the adaptec cards 
configured and installed on, while the megaraid was a snap.  Installed RH, 
installed the dkms rpm, installed the dkms enabled megaraid driver and 
rebooted.  Literally, that's all it took.




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
@ 2004-05-11 21:11       ` Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:29         ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 21:42         ` Re: Quad processor options Rob Sell <lists@facnd.com>
  1 sibling, 2 replies; 25+ messages in thread

From: Bjoern Metzdorf @ 2004-05-11 21:11 UTC (permalink / raw)
  To: scott.marlowe <scott.marlowe@ihs.com>; +Cc: pgsql-performance@postgresql.org, "Pgsql-Admin (E-mail)" <pgsql-admin@postgresql.org>

scott.marlowe wrote:
>>Next drives I'll buy will certainly be 15k scsi drives.
> 
> Better to buy more 10k drives than fewer 15k drives.  Other than slightly 
> faster select times, the 15ks aren't really any faster.

Good to know. I'll remember that.

>>In peak times we can get up to 700-800 connections at the same time. 
>>There are quite some updates involved, without having exact numbers I'll 
>>think that we have about 70% selects and 30% updates/inserts.
> 
> Wow, a lot of writes then.

Yes, it certainly could also be only 15-20% updates/inserts, but this is 
also not negligible.

> Sure, adaptec makes one, so does lsi megaraid.  Dell resells both of 
> these, the PERC3DI and the PERC3DC are adaptec, then lsi in that order, I 
> believe.  We run the lsi megaraid with 64 megs battery backed cache.

The LSI sounds good.

> Intel also makes one, but I've heard nothing about it.

It could well be the ICP Vortex one, ICP was bought by Intel some time ago..

> I haven't directly tested anything but the adaptec and the lsi megaraid.  
> Here at work we've had massive issues trying to get the adaptec cards 
> configured and installed on, while the megaraid was a snap.  Installed RH, 
> installed the dkms rpm, installed the dkms enabled megaraid driver and 
> rebooted.  Literally, that's all it took.

I didn't hear anything about dkms for debian, so I will be hand-patching 
as usual :)

Regards,
Bjoern




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 21:11       ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 21:29         ` scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 22:13           ` Re: Quad processor options Allan Wind <allanwind@lifeintegrity.com>
  1 sibling, 1 reply; 25+ messages in thread

From: scott.marlowe @ 2004-05-11 21:29 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: pgsql-performance; Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

On Tue, 11 May 2004, Bjoern Metzdorf wrote:

> scott.marlowe wrote:
> > Sure, adaptec makes one, so does lsi megaraid.  Dell resells both of 
> > these, the PERC3DI and the PERC3DC are adaptec, then lsi in that order, I 
> > believe.  We run the lsi megaraid with 64 megs battery backed cache.
> 
> The LSI sounds good.
> 
> > Intel also makes one, but I've heard nothing about it.
> 
> It could well be the ICP Vortex one, ICP was bought by Intel some time ago..

Also, there are bigger, faster external RAID boxes as well, that make the 
internal cards seem puny.  They're nice because all you need in your main 
box is a good U320 controller to plug into the external RAID array.

That URL I mentioned earlier that had prices has some of the external 
boxes listed.  No price, not for sale on the web, get out the checkbook 
and write a blank check is my guess.  I.e. they're not cheap.

The other nice thing about the LSI cards is that you can install >1 and 
the act like one big RAID array.  i.e. install two cards with a 20 drive 
RAID0 then make a RAID1 across them, and if one or the other cards itself 
fails, you've still got 100% of your data sitting there.  Nice to know you 
can survive the complete failure of one half of your chain.

> > I haven't directly tested anything but the adaptec and the lsi megaraid.  
> > Here at work we've had massive issues trying to get the adaptec cards 
> > configured and installed on, while the megaraid was a snap.  Installed RH, 
> > installed the dkms rpm, installed the dkms enabled megaraid driver and 
> > rebooted.  Literally, that's all it took.
> 
> I didn't hear anything about dkms for debian, so I will be hand-patching 
> as usual :)

Yeah, it seems to be an RPM kinda thing.  But, I'm thinking the 2.0 
drivers got included in the latest 2.6 kernels, so no biggie. I was 
looking around in google, and it definitely appears the 2.x and 1.x 
megaraid drivers were merged into "unified" driver in 2.6 kernel.




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 21:11       ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:29         ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
@ 2004-05-11 22:13           ` Allan Wind <allanwind@lifeintegrity.com>
  2004-05-11 22:44             ` Re: Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  0 siblings, 1 reply; 25+ messages in thread

From: allanwind@lifeintegrity.com @ 2004-05-11 22:13 UTC (permalink / raw)
  To: pgsql-performance

On 2004-05-11T15:29:46-0600, scott.marlowe wrote:
> The other nice thing about the LSI cards is that you can install >1 and 
> the act like one big RAID array.  i.e. install two cards with a 20 drive 
> RAID0 then make a RAID1 across them, and if one or the other cards itself 
> fails, you've still got 100% of your data sitting there.  Nice to know you 
> can survive the complete failure of one half of your chain.

... unless that dying controller corrupted your file system.  Depending
on your tolerance for risk, you may not want to operate for long with a
file system in an unknown state.

Btw, the Intel and LSI Logic RAID controller cards have suspeciously
similar specificationsi, so I would be surprised if one is an OEM.


/Allan
-- 
Allan Wind
P.O. Box 2022
Woburn, MA 01888-0022
USA

Attachments:

  signature.asc (application/pgp-signature, 188B) download

^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 21:11       ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:29         ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 22:13           ` Re: Quad processor options Allan Wind <allanwind@lifeintegrity.com>
@ 2004-05-11 22:44             ` scott.marlowe <scott.marlowe@ihs.com>
  0 siblings, 0 replies; 25+ messages in thread

From: scott.marlowe @ 2004-05-11 22:44 UTC (permalink / raw)
  To: Allan Wind <allanwind@lifeintegrity.com>; +Cc: pgsql-performance

On Tue, 11 May 2004, Allan Wind wrote:

> On 2004-05-11T15:29:46-0600, scott.marlowe wrote:
> > The other nice thing about the LSI cards is that you can install >1 and 
> > the act like one big RAID array.  i.e. install two cards with a 20 drive 
> > RAID0 then make a RAID1 across them, and if one or the other cards itself 
> > fails, you've still got 100% of your data sitting there.  Nice to know you 
> > can survive the complete failure of one half of your chain.
> 
> ... unless that dying controller corrupted your file system.  Depending
> on your tolerance for risk, you may not want to operate for long with a
> file system in an unknown state.

It would have to be the primary controller for that to happen.  The way 
the LSI's work is that you disable the BIOS on the 2nd to 4th cards, and 
the first card, with the active BIOS acts as the primary controller.

In this case, that means the main card is doing the RAID1 work, then 
handing off the data to the subordinate cards.

The subordinate cards do all their own RAID0 work.

mobo ---controller 1--<array1 of disks in RAID0
.....|--controller 2--<array2 of disks in RAID0

and whichever controller fails just kind of disappears.

Note that if it is the master controller, then you'll have to shut down 
and enable the BIOS on one of the secondardy (now primary) controllers.

So while it's possible for the master card failing to corrupt the RAID1 
set, it's still a more reliable system that with just one card.

But nothing is 100% reliable, sadly.

> Btw, the Intel and LSI Logic RAID controller cards have suspeciously
> similar specificationsi, so I would be surprised if one is an OEM.

Hmmm.  I'll take a closer look.




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 21:11       ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 21:42         ` Rob Sell <lists@facnd.com>
  1 sibling, 0 replies; 25+ messages in thread

From: Rob Sell @ 2004-05-11 21:42 UTC (permalink / raw)
  To: pgsql-performance


-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Bjoern Metzdorf
Sent: Tuesday, May 11, 2004 3:11 PM
To: scott.marlowe
Cc: pgsql-performance@postgresql.org; Pgsql-Admin (E-mail)
Subject: Re: [PERFORM] Quad processor options

scott.marlowe wrote:
>>Next drives I'll buy will certainly be 15k scsi drives.
> 
> Better to buy more 10k drives than fewer 15k drives.  Other than slightly 
> faster select times, the 15ks aren't really any faster.

Good to know. I'll remember that.

>>In peak times we can get up to 700-800 connections at the same time. 
>>There are quite some updates involved, without having exact numbers I'll 
>>think that we have about 70% selects and 30% updates/inserts.
> 
> Wow, a lot of writes then.

Yes, it certainly could also be only 15-20% updates/inserts, but this is 
also not negligible.

> Sure, adaptec makes one, so does lsi megaraid.  Dell resells both of 
> these, the PERC3DI and the PERC3DC are adaptec, then lsi in that order, I 
> believe.  We run the lsi megaraid with 64 megs battery backed cache.

The LSI sounds good.

> Intel also makes one, but I've heard nothing about it.

It could well be the ICP Vortex one, ICP was bought by Intel some time ago..

> I haven't directly tested anything but the adaptec and the lsi megaraid.  
> Here at work we've had massive issues trying to get the adaptec cards 
> configured and installed on, while the megaraid was a snap.  Installed RH,

> installed the dkms rpm, installed the dkms enabled megaraid driver and 
> rebooted.  Literally, that's all it took.

I didn't hear anything about dkms for debian, so I will be hand-patching 
as usual :)

Regards,
Bjoern


-------------------------

Personally I would stay away from anything intel over 2 processors.  I have
done some research and if memory serves it something like this. Intel's
architecture makes each processor compete for bandwidth on the bus to the
ram. Amd differs in that each proc has its own bus to the ram.

Don't take this as god's honest fact but just keep it in mind when
considering a Xeon solution, it may be worth your time to do some deeper
research into this. There is some on this here
http://www4.tomshardware.com/cpu/20030422/ 

Rob




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
@ 2004-05-12 04:58       ` Grega Bremec <grega.bremec@noviforum.si>
  2004-05-12 13:44         ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  1 sibling, 1 reply; 25+ messages in thread

From: Grega Bremec @ 2004-05-12 04:58 UTC (permalink / raw)
  To: scott.marlowe <scott.marlowe@ihs.com>; +Cc: Bjoern Metzdorf <bm@turtle-entertainment.de>; pgsql-performance@postgresql.org, "Pgsql-Admin (E-mail)" <pgsql-admin@postgresql.org>

...and on Tue, May 11, 2004 at 03:02:24PM -0600, scott.marlowe used the keyboard:
> 
> If you get the LSI megaraid, make sure you're running the latest megaraid 
> 2 driver, not the older, slower 1.18 series.  If you are running linux, 
> look for the dkms packaged version.  dkms, (Dynamic Kernel Module System) 
> automagically compiles and installs source rpms for drivers when you 
> install them, and configures the machine to use them to boot up.  Most 
> drivers seem to be slowly headed that way in the linux universe, and I 
> really like the simplicity and power of dkms.
> 

Hi,

Given the fact LSI MegaRAID seems to be a popular solution around here, and
many of you folx use Linux as well, I thought sharing this piece of info
might be of use.

Running v2 megaraid driver on a 2.4 kernel is actually not a good idea _at_
_all_, as it will silently corrupt your data in the event of a disk failure.

Sorry to have to say so, but we tested it (on kernels up to 2.4.25, not sure
about 2.4.26 yet) and it comes out it doesn't do hotswap the way it should.

Somehow the replaced disk drives are not _really_ added to the array, which
continues to work in degraded mode for a while and (even worse than that)
then starts to think the replaced disk is in order without actually having
resynced it, thus beginning to issue writes to non-existant areas of it.

The 2.6 megaraid driver indeed seems to be a merged version of the above
driver and the old one, giving both improved performance and correct
functionality in the event of a hotswap taking place.

Hope this helped,
-- 
    Grega Bremec
    Senior Administrator
    Noviforum Ltd., Software & Media
    http://www.noviforum.si/

^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 21:02     ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-12 04:58       ` Re: [PERFORM] Quad processor options Grega Bremec <grega.bremec@noviforum.si>
@ 2004-05-12 13:44         ` scott.marlowe <scott.marlowe@ihs.com>
  0 siblings, 0 replies; 25+ messages in thread

From: scott.marlowe @ 2004-05-12 13:44 UTC (permalink / raw)
  To: Grega Bremec <grega.bremec@noviforum.si>; +Cc: Bjoern Metzdorf <bm@turtle-entertainment.de>; pgsql-performance; Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

On Wed, 12 May 2004, Grega Bremec wrote:

> ...and on Tue, May 11, 2004 at 03:02:24PM -0600, scott.marlowe used the keyboard:
> > 
> > If you get the LSI megaraid, make sure you're running the latest megaraid 
> > 2 driver, not the older, slower 1.18 series.  If you are running linux, 
> > look for the dkms packaged version.  dkms, (Dynamic Kernel Module System) 
> > automagically compiles and installs source rpms for drivers when you 
> > install them, and configures the machine to use them to boot up.  Most 
> > drivers seem to be slowly headed that way in the linux universe, and I 
> > really like the simplicity and power of dkms.
> > 
> 
> Hi,
> 
> Given the fact LSI MegaRAID seems to be a popular solution around here, and
> many of you folx use Linux as well, I thought sharing this piece of info
> might be of use.
> 
> Running v2 megaraid driver on a 2.4 kernel is actually not a good idea _at_
> _all_, as it will silently corrupt your data in the event of a disk failure.
> 
> Sorry to have to say so, but we tested it (on kernels up to 2.4.25, not sure
> about 2.4.26 yet) and it comes out it doesn't do hotswap the way it should.
> 
> Somehow the replaced disk drives are not _really_ added to the array, which
> continues to work in degraded mode for a while and (even worse than that)
> then starts to think the replaced disk is in order without actually having
> resynced it, thus beginning to issue writes to non-existant areas of it.
> 
> The 2.6 megaraid driver indeed seems to be a merged version of the above
> driver and the old one, giving both improved performance and correct
> functionality in the event of a hotswap taking place.

This doesn't make any sense to me, since the hot swapping is handled by 
the card autonomously.  I also tested it with a hot spare and pulled one 
drive and it worked fine during our acceptance testing.

However, I've got a hot spare machine I can test on, so I'll try it again 
and see if I can make it fail.

when testing it, was the problem present in certain RAID configurations or 
only one type or what?  I'm curious to try and reproduce this problem, 
since I've never heard of it before.

Also, what firmware version were those megaraid cards, ours is fairly 
new, as we got it at the beginning of this year, and I'm wondering if it 
is a firmware issue.




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 22:46     ` Paul Tuckfield <paul@tuckfield.com>
  2004-05-12 08:58       ` Re: Quad processor options Andrew McMillan <andrew@catalyst.net.nz>
  2004-05-12 10:17       ` Re: [PERFORM] Quad processor options Manfred Koizar <mkoi-pg@aon.at>
  1 sibling, 2 replies; 25+ messages in thread

From: Paul Tuckfield @ 2004-05-11 22:46 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>; scott.marlowe <scott.marlowe@ihs.com>; pgsql-performance

I'm confused why you say the system is 70% busy: the vmstat output 
shows 70% *idle*.

The vmstat you sent shows good things and ambiguous things:
- si and so are zero, so your not paging/swapping.  Thats always step 
1.  you're fine.
- bi and bo (physical IO) shows pretty high numbers for how many disks 
you have.
   (assuming random IO) so please send an "iostat 10" sampling during 
peak.
- note that cpu is only 30% busy.  that should mean that adding cpus 
will *not* help.
- the "cache" column shows that linux is using 2.3G for cache. (way too 
much)
   you generally want to give memory to postgres to keep it "close" to 
the user,
   not leave it unused to be claimed by linux cache (need to leave 
*some* for linux tho)

My recommendations:
- I'll bet you have a low value for shared buffers, like 10000.  On 
your 3G system
   you should ramp up the value to at least 1G (125000 8k buffers) 
unless something
   else runs on the system.   It's best to not do things too 
drastically, so if Im right and
   you sit at 10000 now, try going to 30000 then 60000 then 125000 or 
above.

- if the above is off base, then I wonder why we see high runque 
numbers in spite
   of over 60% idle cpu.   Maybe some serialization happening somewhere. 
  Also depending
   on how you've laid out your 4 disk drives, you may see all IOs going 
to one drive. the 7M/sec
   is on the high side, if that's the case.  iostat numbers will reveal 
if it's skewed, and if it's random,
  tho linux iostat doesn't seem to report response times (sigh)   
Response times are the golden
  metric when diagnosing IO thruput in OLTP / stripe situation.




On May 11, 2004, at 1:41 PM, Bjoern Metzdorf wrote:

> scott.marlowe wrote:
>
>> Well, from what I've read elsewhere on the internet, it would seem 
>> the Opterons scale better to 4 CPUs than the basic Xeons do.  Of 
>> course, the exception to this is SGI's altix, which uses their own 
>> chipset and runs the itanium with very good memory bandwidth.
>
> This is basically what I read too. But I cannot spent money on a quad 
> opteron just for testing purposes :)
>
>> But, do you really need more CPU horsepower?
>> Are you I/O or CPU or memory or memory bandwidth bound?  If you're 
>> sitting at 99% idle, and iostat says your drives are only running at 
>> some small percentage of what you know they could, you might be 
>> memory or memory bandwidth limited.  Adding two more CPUs will not 
>> help with that situation.
>
> Right now we have a dual xeon 2.4, 3 GB Ram, Mylex extremeraid 
> controller, running 2 Compaq BD018122C0, 1 Seagate ST318203LC and 1 
> Quantum ATLAS_V_18_SCA.
>
> iostat show between 20 and 60 % user avg-cpu. And this is not even 
> peak time.
>
> I attached a "vmstat 10 120" output for perhaps 60-70% peak load.
>
>> If your I/O is saturated, then the answer may well be a better RAID 
>> array, with many more drives plugged into it.  Do you have any spare 
>> drives you can toss on the machine to see if that helps?  Sometimes 
>> going from 4 drives in a RAID 1+0 to 6 or 8 or more can give a big 
>> boost in performance.
>
> Next drives I'll buy will certainly be 15k scsi drives.
>
>> In short, don't expect 4 CPUs to solve the problem if the problem 
>> isn't really the CPUs being maxed out.
>> Also, what type of load are you running?  Mostly read, mostly 
>> written, few connections handling lots of data, lots of connections 
>> each handling a little data, lots of transactions, etc...
>
> In peak times we can get up to 700-800 connections at the same time. 
> There are quite some updates involved, without having exact numbers 
> I'll think that we have about 70% selects and 30% updates/inserts.
>
>> If you are doing lots of writing, make SURE you have a controller 
>> that supports battery backed cache and is configured to write-back, 
>> not write-through.
>
> Could you recommend a certain controller type? The only battery backed 
> one that I found on the net is the newest model from icp-vortex.com.
>
> Regards,
> Bjoern
> ~# vmstat 10 120
>    procs                      memory    swap          io     system    
>      cpu
>  r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  
> us  sy  id
>  1  1  0  24180  10584  32468 2332208   0   1     0     2    1     2   
> 2   0   0
>  0  2  0  24564  10480  27812 2313528   8   0  7506   574 1199  8674  
> 30   7  63
>  2  1  0  24692  10060  23636 2259176   0  18  8099   298 2074  6328  
> 25   7  68
>  2  0  0  24584  18576  21056 2299804   3   6 13208   305 1598  8700  
> 23   6  71
>  1 21  1  24504  16588  20912 2309468   4   0  1442  1107  754  6874  
> 42  13  45
>  6  1  0  24632  13148  19992 2319400   0   0  2627   499 1184  9633  
> 37   6  58
>  5  1  0  24488  10912  19292 2330080   5   0  3404   150 1466 10206  
> 32   6  61
>  4  1  0  24488  12180  18824 2342280   3   0  2934    40 1052  3866  
> 19   3  78
>  0  0  0  24420  14776  19412 2347232   6   0   403   216 1123  4702  
> 22   3  74
>  0  0  0  24548  14408  17380 2321780   4   0   522   715  965  6336  
> 25   5  71
>  4  0  0  24676  12504  17756 2322988   0   0   564   830  883  7066  
> 31   6  63
>  0  3  0  24676  14060  18232 2325224   0   0   483   388 1097  3401  
> 21   3  76
>  0  2  1  24676  13044  18700 2322948   0   0   701   195 1078  5187  
> 23   3  74
>  2  0  0  24676  21576  18752 2328168   0   0   467   177 1552  3574  
> 18   3  78
>
> ---------------------------(end of 
> broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to 
> majordomo@postgresql.org)




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 22:46     ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
@ 2004-05-12 08:58       ` Andrew McMillan <andrew@catalyst.net.nz>
  1 sibling, 0 replies; 25+ messages in thread

From: Andrew McMillan @ 2004-05-12 08:58 UTC (permalink / raw)
  To: Paul Tuckfield <paul@tuckfield.com>; +Cc: Bjoern Metzdorf <bm@turtle-entertainment.de>; pgsql-performance

On Tue, 2004-05-11 at 15:46 -0700, Paul Tuckfield wrote:

> - the "cache" column shows that linux is using 2.3G for cache. (way too 
> much) you generally want to give memory to postgres to keep it "close" to 
> the user, not leave it unused to be claimed by linux cache (need to leave 
> *some* for linux tho)
> 
> My recommendations:
> - I'll bet you have a low value for shared buffers, like 10000.  On 
> your 3G system you should ramp up the value to at least 1G (125000 8k buffers) 
> unless something else runs on the system.   It's best to not do things too 
> drastically, so if Im right and you sit at 10000 now, try going to
> 30000 then 60000 then 125000 or above.

Huh?

Doesn't this run counter to almost every piece of PostgreSQL performance
tuning advice given?

I run my own boxes with buffers set to around 10000-20000 and an
effective_cache_size = 375000 (8k pages - around 3G).

That's working well with PostgreSQL 7.4.2 under Debian "woody" (using
Oliver Elphick's backported packages from
http://people.debian.org/~elphick/debian/).

Regards,
					Andrew.
-------------------------------------------------------------------------
Andrew @ Catalyst .Net .NZ  Ltd,  PO Box 11-053,  Manners St,  Wellington
WEB: http://catalyst.net.nz/             PHYS: Level 2, 150-154 Willis St
DDI: +64(4)916-7201       MOB: +64(21)635-694      OFFICE: +64(4)499-2267
Q:        How much does it cost to ride the Unibus?
A:        2 bits.
-------------------------------------------------------------------------




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 22:46     ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
@ 2004-05-12 10:17       ` Manfred Koizar <mkoi-pg@aon.at>
  2004-05-12 10:27         ` Re: [PERFORM] Quad processor options Halford Dace <hal@stowe.co.za>
  1 sibling, 1 reply; 25+ messages in thread

From: Manfred Koizar @ 2004-05-12 10:17 UTC (permalink / raw)
  To: Paul Tuckfield <paul@tuckfield.com>; +Cc: Bjoern Metzdorf <bm@turtle-entertainment.de>; Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>; scott.marlowe <scott.marlowe@ihs.com>; pgsql-performance

On Tue, 11 May 2004 15:46:25 -0700, Paul Tuckfield <paul@tuckfield.com>
wrote:
>- the "cache" column shows that linux is using 2.3G for cache. (way too 
>much)

There is no such thing as "way too much cache".

>   you generally want to give memory to postgres to keep it "close" to 
>the user,

Yes, but only a moderate amount of memory.

>   not leave it unused to be claimed by linux cache

Cache is not unused memory.

>- I'll bet you have a low value for shared buffers, like 10000.  On 
>your 3G system
>   you should ramp up the value to at least 1G (125000 8k buffers) 

In most cases this is almost the worst thing you can do.  The only thing
even worse would be setting it to 1.5 G.

Postgres is just happy with a moderate shared_buffers setting.  We
usually recommend something like 10000.  You could try 20000, but don't
increase it beyond that without strong evidence that it helps in your
particular case.

This has been discussed several times here, on -hackers and on -general.
Search the archives for more information.

Servus
 Manfred



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 20:16 ` Re: [PERFORM] Quad processor options scott.marlowe <scott.marlowe@ihs.com>
  2004-05-11 20:41   ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  2004-05-11 22:46     ` Re: [PERFORM] Quad processor options Paul Tuckfield <paul@tuckfield.com>
  2004-05-12 10:17       ` Re: [PERFORM] Quad processor options Manfred Koizar <mkoi-pg@aon.at>
@ 2004-05-12 10:27         ` Halford Dace <hal@stowe.co.za>
  0 siblings, 0 replies; 25+ messages in thread

From: Halford Dace @ 2004-05-12 10:27 UTC (permalink / raw)
  To: pgsql-admin@postgresql.org


On 12 May 2004, at 12:17 PM, Manfred Koizar wrote:

> On Tue, 11 May 2004 15:46:25 -0700, Paul Tuckfield <paul@tuckfield.com>
> wrote:
>
>> - I'll bet you have a low value for shared buffers, like 10000.  On
>> your 3G system
>>   you should ramp up the value to at least 1G (125000 8k buffers)
>
> In most cases this is almost the worst thing you can do.  The only 
> thing
> even worse would be setting it to 1.5 G.
>
> Postgres is just happy with a moderate shared_buffers setting.  We
> usually recommend something like 10000.  You could try 20000, but don't
> increase it beyond that without strong evidence that it helps in your
> particular case.
>
> This has been discussed several times here, on -hackers and on 
> -general.
> Search the archives for more information.

We have definitely found this to be true here.  We have some fairly 
complex queries running on a rather underpowered box (beautiful but 
steam-driven old Silicon Graphics Challenge DM).  We ended up using a 
very slight increase to shared buffers, but gaining ENORMOUSLY through 
proper optimisation of queries, appropriate indices and the use of 
optimizer-bludgeons like "SET ENABLE_SEQSCAN = OFF"

Hal




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-11 22:23 ` J. Andrew Rogers <jrogers@neopolitan.com>
  3 siblings, 0 replies; 25+ messages in thread

From: J. Andrew Rogers @ 2004-05-11 22:23 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; pgsql-performance

On Tue, 2004-05-11 at 12:06, Bjoern Metzdorf wrote:
> Has anyone experiences with quad Xeon or quad Opteron setups? I am 
> looking at the appropriate boards from Tyan, which would be the only 
> option for us to buy such a beast. The 30k+ setups from Dell etc. don't 
> fit our budget.
> 
> I am thinking of the following:
> 
> Quad processor (xeon or opteron)
> 5 x SCSI 15K RPM for Raid 10 + spare drive
> 2 x IDE for system
> ICP-Vortex battery backed U320 Hardware Raid
> 4-8 GB Ram


Just to add my two cents to the fray:

We use dual Opterons around here and prefer them to the Xeons for
database servers.  As others have pointed out, the Opteron systems will
scale well to more than two processors unlike the Xeon.  I know a couple
people with quad Opterons and it apparently scales very nicely, unlike
quad Xeons which don't give you much more.  On some supercomputing
hardware lists I'm on, they seem to be of the opinion that the current
Opteron fabric won't really show saturation until you have 6-8 CPUs
connected to it.

Like the other folks said, skip the 15k drives.  Those will only give
you a marginal improvement for an integer factor price increase over 10k
drives.  Instead spend your money on a nice RAID controller with a fat
cache and a backup battery, and maybe some extra spindles for your
array.  I personally like the LSI MegaRAID 320-2, which I always max out
to 256Mb of cache RAM and the required battery.  A maxed out LSI 320-2
should set you back <$1k.  Properly configured, you will notice large
improvements in the performance of your disk subsystem, especially if
you have a lot of writing going on.

I would recommend getting the Opterons, and spending the relatively
modest amount of money to get nice RAID controller with a large
write-back cache while sticking with 10k drives.

Depending on precisely how you configure it, this should cost you no
more than $10-12k.  We just built a very similar configuration, but with
dual Opterons on an HDAMA motherboard rather than a quad Tyan, and it
cost <$6k inclusive of everything.  Add the money for 4 of the 8xx
processors and the Tyan quad motherboard, and the sum comes out to a
very reasonable number for what you are getting.


j. andrew rogers






^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
@ 2004-05-12 04:03 ` Dennis Bjorklund <db@zigo.dhs.org>
  3 siblings, 0 replies; 25+ messages in thread

From: Dennis Bjorklund @ 2004-05-12 04:03 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: pgsql-performance@postgresql.org, "Pgsql-Admin (E-mail)" <pgsql-admin@postgresql.org>

On Tue, 11 May 2004, Bjoern Metzdorf wrote:

> I am curious if there are any real life production quad processor setups 
> running postgresql out there. Since postgresql lacks a proper 
> replication/cluster solution, we have to buy a bigger machine.

Du you run the latest version of PG? I've read the thread bug have not 
seen any information about what pg version. All I've seen was a reference 
to debian which might just as well mean that you run pg 7.2 (probably not 
but I have to ask).

Some classes of queries run much faster in pg 7.4 then in older versions
so if you are lucky that can help.

-- 
/Dennis Björklund




^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
@ 2004-05-11 19:32 Anjan Dave <adave@vantage.com>
  2004-05-11 20:28 ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  0 siblings, 1 reply; 25+ messages in thread

From: Anjan Dave @ 2004-05-11 19:32 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; pgsql-performance; +Cc: Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

We use XEON Quads (PowerEdge 6650s) and they work nice, provided you configure the postgres properly. Dell is the cheapest quad you can buy i think. You shouldn't be paying 30K unless you are getting high CPU-cache on each processor and tons of memory.
 
I am actually curious, have you researched/attempted any postgresql clustering solutions? I agree, you can't just keep buying bigger machines.
 
They have 5 internal drives (4 in RAID 10, 1 spare) on U320, 128MB cache on the PERC controller, 8GB RAM.
 
Thanks,
Anjan

	-----Original Message----- 
	From: Bjoern Metzdorf [mailto:bm@turtle-entertainment.de] 
	Sent: Tue 5/11/2004 3:06 PM 
	To: pgsql-performance@postgresql.org 
	Cc: Pgsql-Admin (E-mail) 
	Subject: [PERFORM] Quad processor options
	
	

	Hi,
	
	I am curious if there are any real life production quad processor setups
	running postgresql out there. Since postgresql lacks a proper
	replication/cluster solution, we have to buy a bigger machine.
	
	Right now we are running on a dual 2.4 Xeon, 3 GB Ram and U160 SCSI
	hardware-raid 10.
	
	Has anyone experiences with quad Xeon or quad Opteron setups? I am
	looking at the appropriate boards from Tyan, which would be the only
	option for us to buy such a beast. The 30k+ setups from Dell etc. don't
	fit our budget.
	
	I am thinking of the following:
	
	Quad processor (xeon or opteron)
	5 x SCSI 15K RPM for Raid 10 + spare drive
	2 x IDE for system
	ICP-Vortex battery backed U320 Hardware Raid
	4-8 GB Ram
	
	Would be nice to hear from you.
	
	Regards,
	Bjoern
	
	---------------------------(end of broadcast)---------------------------
	TIP 4: Don't 'kill -9' the postmaster
	



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 19:32 Re: [PERFORM] Quad processor options Anjan Dave <adave@vantage.com>
@ 2004-05-11 20:28 ` Bjoern Metzdorf <bm@turtle-entertainment.de>
  0 siblings, 0 replies; 25+ messages in thread

From: Bjoern Metzdorf @ 2004-05-11 20:28 UTC (permalink / raw)
  To: Anjan Dave <adave@vantage.com>; +Cc: pgsql-performance@postgresql.org, "Pgsql-Admin (E-mail)" <pgsql-admin@postgresql.org>

Anjan Dave wrote:

> We use XEON Quads (PowerEdge 6650s) and they work nice,
 > provided you configure the postgres properly.
 > Dell is the cheapest quad you can buy i think.
 > You shouldn't be paying 30K unless you are getting high CPU-cache
 > on each processor and tons of memory.

good to hear, I tried to online configure a quad xeon here at dell 
germany, but the 6550 is not available for online configuration. at dell 
usa it works. I will give them a call tomorrow.

> I am actually curious, have you researched/attempted any 
 > postgresql clustering solutions?
 > I agree, you can't just keep buying bigger machines.

There are many asynchronous, trigger based solutions out there (eRserver 
etc..), but what we need is basically a master <-> master setup, which 
seems not to be available soon for postgresql.

Our current dual Xeon runs at 60-70% average cpu load, which is really 
much. I cannot afford any trigger overhead here. This machine is 
responsible for over 30M page impressions per month, 50 page impressums 
per second at peak times. The autovacuum daemon is a god sent gift :)

I'm curious how the recently announced mysql cluster will perform, 
although it is not an option for us. postgresql has far superior 
functionality.

> They have 5 internal drives (4 in RAID 10, 1 spare) on U320, 
 > 128MB cache on the PERC controller, 8GB RAM.

Could you tell me what you paid approximately for this setup?

How does it perform? It certainly won't be twice as fast a as dual xeon, 
but I remember benchmarking a quad P3 xeon some time ago, and it was 
disappointingly slow...

Regards,
Bjoern



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
@ 2004-05-11 20:38 Anjan Dave <adave@vantage.com>
  2004-05-11 20:45 ` Re: [PERFORM] Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
  0 siblings, 1 reply; 25+ messages in thread

From: Anjan Dave @ 2004-05-11 20:38 UTC (permalink / raw)
  To: Bjoern Metzdorf <bm@turtle-entertainment.de>; +Cc: pgsql-performance; Pgsql-Admin (E-mail) <pgsql-admin@postgresql.org>

Did you mean to say the trigger-based clustering solution is loading the dual CPUs 60-70% right now?
 
Performance will not be linear with more processors, but it does help with more processes. We haven't benchmarked it, but we haven't had any problems also so far in terms of performance.
 
Price would vary with your relation/yearly purchase, etc, but a 6650 with 2.0GHz/1MB cache/8GB Memory, RAID card, drives, etc, should definitely cost you less than 20K USD.
 
-anjan

	-----Original Message----- 
	From: Bjoern Metzdorf [mailto:bm@turtle-entertainment.de] 
	Sent: Tue 5/11/2004 4:28 PM 
	To: Anjan Dave 
	Cc: pgsql-performance@postgresql.org; Pgsql-Admin (E-mail) 
	Subject: Re: [PERFORM] Quad processor options
	
	

	Anjan Dave wrote:
	
	> We use XEON Quads (PowerEdge 6650s) and they work nice,
	 > provided you configure the postgres properly.
	 > Dell is the cheapest quad you can buy i think.
	 > You shouldn't be paying 30K unless you are getting high CPU-cache
	 > on each processor and tons of memory.
	
	good to hear, I tried to online configure a quad xeon here at dell
	germany, but the 6550 is not available for online configuration. at dell
	usa it works. I will give them a call tomorrow.
	
	> I am actually curious, have you researched/attempted any
	 > postgresql clustering solutions?
	 > I agree, you can't just keep buying bigger machines.
	
	There are many asynchronous, trigger based solutions out there (eRserver
	etc..), but what we need is basically a master <-> master setup, which
	seems not to be available soon for postgresql.
	
	Our current dual Xeon runs at 60-70% average cpu load, which is really
	much. I cannot afford any trigger overhead here. This machine is
	responsible for over 30M page impressions per month, 50 page impressums
	per second at peak times. The autovacuum daemon is a god sent gift :)
	
	I'm curious how the recently announced mysql cluster will perform,
	although it is not an option for us. postgresql has far superior
	functionality.
	
	> They have 5 internal drives (4 in RAID 10, 1 spare) on U320,
	 > 128MB cache on the PERC controller, 8GB RAM.
	
	Could you tell me what you paid approximately for this setup?
	
	How does it perform? It certainly won't be twice as fast a as dual xeon,
	but I remember benchmarking a quad P3 xeon some time ago, and it was
	disappointingly slow...
	
	Regards,
	Bjoern
	



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: [PERFORM] Quad processor options
  2004-05-11 20:38 Re: [PERFORM] Quad processor options Anjan Dave <adave@vantage.com>
@ 2004-05-11 20:45 ` Bjoern Metzdorf <bm@turtle-entertainment.de>
  0 siblings, 0 replies; 25+ messages in thread

From: Bjoern Metzdorf @ 2004-05-11 20:45 UTC (permalink / raw)
  To: Anjan Dave <adave@vantage.com>; +Cc: pgsql-performance@postgresql.org, "Pgsql-Admin (E-mail)" <pgsql-admin@postgresql.org>

Anjan Dave wrote:

> Did you mean to say the trigger-based clustering solution 
 > is loading the dual CPUs 60-70% right now?

No, this is without any triggers involved.

> Performance will not be linear with more processors, 
 > but it does help with more processes.
 > We haven't benchmarked it, but we haven't had any
 > problems also so far in terms of performance.

 From the amount of processes view, we certainly can saturate a quad 
setup :)

> Price would vary with your relation/yearly purchase, etc, 
 > but a 6650 with 2.0GHz/1MB cache/8GB Memory, RAID card,
 > drives, etc, should definitely cost you less than 20K USD.

Which is still very much. Anyone have experience with a self built quad 
xeon, using the Tyan Thunder board?

Regards,
Bjoern



^ permalink  raw  reply  [nested|flat] 25+ messages in thread

* Re: Quad processor options
@ 2004-05-19 03:38 Marty Scholes <marty@outputservices.com>
  0 siblings, 0 replies; 25+ messages in thread

From: Marty Scholes @ 2004-05-19 03:38 UTC (permalink / raw)
  To: pgsql-performance

After reading the replies to this, it is clear that this is a 
Lintel-centric question, but I will throw in my experience.

 > I am curious if there are any real life production
 > quad processor setups running postgresql out there.

Yes.  We are running a 24/7 operation on a quad CPU Sun V880.

 > Since postgresql lacks a proper replication/cluster
 > solution, we have to buy a bigger machine.

This was a compelling reason for us to stick with SPARC and avoid 
Intel/AMD when picking a DB server.  We moved off of an IBM mainframe in 
1993 to Sun gear and never looked back.  We can upgrade to our heart's 
content with minimal disruption and are only on our third box in 11 
years with plenty of life left in our current one.

 > Right now we are running on a dual 2.4 Xeon, 3 GB Ram
 > and U160 SCSI hardware-raid 10.

A couple people mentioned hardware RAID, which I completely agree with. 
  I prefer an external box with a SCSI or FC connector.  There are no 
driver issues that way.  We boot from our arrays.

The Nexsan ATABoy2 is a nice blend of performance, reliability and cost. 
  Some of these with 1TB and 2TB of space were recently spotted on ebay 
for under $5k.  We run a VERY random i/o mix on ours and it will 
consistently sustain 15 MB/s in blended read and write i/o, sustaining 
well over 1200 io/s.  These are IDE drives, so they fail more often than 
SCSI, so run RAID1 or RAID5.  The cache on these pretty much eliminates 
the RAID5 penalties.

 > The 30k+ setups from Dell etc. don't fit our budget.

For that kind of money you could get a lower end Sun box (or IBM RS/6000 
I would imagine) and give yourself an astounding amount of headroom for 
future growth.

Sincerely,
Marty




^ permalink  raw  reply  [nested|flat] 25+ messages in thread


end of thread, other threads:[~2004-05-19 03:38 UTC | newest]

Thread overview: 25+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-05-11 19:06 Quad processor options Bjoern Metzdorf <bm@turtle-entertainment.de>
2004-05-11 20:14 ` Paul Tuckfield <paul@tuckfield.com>
2004-05-11 20:46   ` Bjoern Metzdorf <bm@turtle-entertainment.de>
2004-05-12 05:34     ` spied@yandex.ru
2004-05-11 20:16 ` scott.marlowe <scott.marlowe@ihs.com>
2004-05-11 20:41   ` Bjoern Metzdorf <bm@turtle-entertainment.de>
2004-05-11 21:02     ` scott.marlowe <scott.marlowe@ihs.com>
2004-05-11 21:11       ` Bjoern Metzdorf <bm@turtle-entertainment.de>
2004-05-11 21:29         ` scott.marlowe <scott.marlowe@ihs.com>
2004-05-11 22:13           ` Allan Wind <allanwind@lifeintegrity.com>
2004-05-11 22:44             ` scott.marlowe <scott.marlowe@ihs.com>
2004-05-11 21:42         ` Rob Sell <lists@facnd.com>
2004-05-12 04:58       ` Grega Bremec <grega.bremec@noviforum.si>
2004-05-12 13:44         ` scott.marlowe <scott.marlowe@ihs.com>
2004-05-11 22:46     ` Paul Tuckfield <paul@tuckfield.com>
2004-05-12 08:58       ` Andrew McMillan <andrew@catalyst.net.nz>
2004-05-12 10:17       ` Manfred Koizar <mkoi-pg@aon.at>
2004-05-12 10:27         ` Halford Dace <hal@stowe.co.za>
2004-05-11 22:23 ` J. Andrew Rogers <jrogers@neopolitan.com>
2004-05-12 04:03 ` Dennis Bjorklund <db@zigo.dhs.org>
2004-05-11 19:32 Re: [PERFORM] Quad processor options Anjan Dave <adave@vantage.com>
2004-05-11 20:28 ` Bjoern Metzdorf <bm@turtle-entertainment.de>
2004-05-11 20:38 Re: [PERFORM] Quad processor options Anjan Dave <adave@vantage.com>
2004-05-11 20:45 ` Bjoern Metzdorf <bm@turtle-entertainment.de>
2004-05-19 03:38 Re: Quad processor options Marty Scholes <marty@outputservices.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox