pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedOptimizing disk throughput on quad Opteron
10+ messages / 6 participants
[nested] [flat]
* Optimizing disk throughput on quad Opteron
@ 2006-10-21 15:43 John Philips <johnphilips42@yahoo.com>
0 siblings, 4 replies; 10+ messages in thread
From: John Philips @ 2006-10-21 15:43 UTC (permalink / raw)
To: pgsql-performance
Hello,
I'm working out specs for a new database server to be
purchased for our organization. The applications the
server will handle are mainly related to network
operations (monitoring, logging, statistical/trend
reports, etc.). Disk I/O will be especially high with
relation to processing network stats.
You can find a diagram of my initial
spec here:
http://img266.imageshack.us/img266/9171/dbserverdiagramuc3.jpg
Server will be a HP ProLiant DL585 G2 with four
dual-core 2.6GHz processors and 8GB of RAM.
I can always throw in more RAM. I'm trying to find
the most effective way to maximize disk throughput, as
the list archives suggest that it is the choke point
in most cases. I separated the storage into multiple
arrays on multiple controllers, and plan to have 512MB
RAM on each controller using BBWC. The plan is to
utilize multiple tablespaces as well as partitioned
tables when necessary. (Note that the
StorageWorks 4214R enclosure with Ultra3 disks is used
because we already have it lying around.)
I heard some say that the transaction log should be on
it's own array, others say it doesn't hurt to have it
on the same array as the OS. Is it really worthwhile
to put it on it's own array?
Can you guys see any glaring bottlenecks in my layout?
Any other suggestions to offer (throw in more
controllers, different RAID layout, etc.)? Our budget
limit is $50k.
Thanks!
P.S. I know there was a very similar thread started by
Ben Suffolk recently, I'd still like to have your
"eyes of experience" look at my proposed layout :-)
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-21 17:33 Joshua D. Drake <jd@commandprompt.com>
parent: John Philips <johnphilips42@yahoo.com>
3 siblings, 0 replies; 10+ messages in thread
From: Joshua D. Drake @ 2006-10-21 17:33 UTC (permalink / raw)
To: John Philips <johnphilips42@yahoo.com>; +Cc: pgsql-performance
> I heard some say that the transaction log should be on
> it's own array, others say it doesn't hurt to have it
> on the same array as the OS. Is it really worthwhile
> to put it on it's own array?
>
> Can you guys see any glaring bottlenecks in my layout?
> Any other suggestions to offer (throw in more
> controllers, different RAID layout, etc.)? Our budget
> limit is $50k.
You should easily be able to fit in 50k since you already have the
storage device. I would suggest the following:
1. Throw in as much RAM as you can.
2. Yes put the transaction logs on a separate array. There are a couple
of reasons for this:
1. transaction logs are written sequentially so a RAID 1 is enough
2. You don't have to use a journaled fs for the transaction logs so it
is really fast.
3. IIRC the MSA 30 can take 14 drives. Make sure you put in all 14
drives and delegate two of them to hot spare duty.
I actually wonder if you would be better off putting the indexes on
tablespace A and use your core data set on the larger storage works array...
Sincerely,
Joshua D. Drake
Sincerely,
Joshua D. Drake
>
> Thanks!
>
> P.S. I know there was a very similar thread started by
> Ben Suffolk recently, I'd still like to have your
> "eyes of experience" look at my proposed layout :-)
>
>
>
>
>
>
>
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-21 18:17 Ben Suffolk <ben@vanilla.net>
parent: John Philips <johnphilips42@yahoo.com>
3 siblings, 1 reply; 10+ messages in thread
From: Ben Suffolk @ 2006-10-21 18:17 UTC (permalink / raw)
To: John Philips <johnphilips42@yahoo.com>; +Cc: pgsql-performance
> You can find a diagram of my initial
> spec here:
> http://img266.imageshack.us/img266/9171/dbserverdiagramuc3.jpg
>
> Can you guys see any glaring bottlenecks in my layout?
> Any other suggestions to offer (throw in more
> controllers, different RAID layout, etc.)? Our budget
> limit is $50k.
The thing I would ask is would you not be better with SAS drives?
Since the comments on Dell, and the highlighted issues I have been
looking at HP and the the Smart Array P600 controller with 512 BBWC.
Although I am looking to stick with the 8 internal disks, rather than
use external ones.
The HP Smart Array 50 is the external array for SAS drives. Not
really looked into it much though.
Regards
Ben
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-23 12:16 John Philips <johnphilips42@yahoo.com>
parent: Ben Suffolk <ben@vanilla.net>
0 siblings, 1 reply; 10+ messages in thread
From: John Philips @ 2006-10-23 12:16 UTC (permalink / raw)
To: Ben Suffolk <ben@vanilla.net>; +Cc: pgsql-performance
> The thing I would ask is would you not be better
> with SAS drives?
>
> Since the comments on Dell, and the highlighted
> issues I have been
> looking at HP and the the Smart Array P600
> controller with 512 BBWC.
> Although I am looking to stick with the 8 internal
> disks, rather than
> use external ones.
>
> The HP Smart Array 50 is the external array for SAS
> drives. Not
> really looked into it much though.
Ben,
The Smart Array 50 supports a maximum of 10 disks and
has a single I/O module, while the Smart Array 30
supports up to 14 disks and can be configured with a
dual I/O module.
I was under the assumption that SAS runs at the same
speed as Ultra320, in which case the Smart Array 30 is
a better bet...
Thanks for your feedback.
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-23 20:56 Vivek Khera <vivek@khera.org>
parent: John Philips <johnphilips42@yahoo.com>
3 siblings, 1 reply; 10+ messages in thread
From: Vivek Khera @ 2006-10-23 20:56 UTC (permalink / raw)
To: pgsql-performance
On Oct 21, 2006, at 11:43 AM, John Philips wrote:
> Can you guys see any glaring bottlenecks in my layout?
> Any other suggestions to offer (throw in more
> controllers, different RAID layout, etc.)? Our budget
> limit is $50k.
If I had $50k budget, I'd be buying the SunFire X4500 and running
Solaris + ZFS on it. However, you're limited to 2 dual core
Opterons, it seems.
Attachments:
smime.p7s (application/pkcs7-signature, 2.5K) download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-23 20:59 Joshua D. Drake <jd@commandprompt.com>
parent: Vivek Khera <vivek@khera.org>
0 siblings, 1 reply; 10+ messages in thread
From: Joshua D. Drake @ 2006-10-23 20:59 UTC (permalink / raw)
To: Vivek Khera <vivek@khera.org>; +Cc: pgsql-performance
Vivek Khera wrote:
>
> On Oct 21, 2006, at 11:43 AM, John Philips wrote:
>
>> Can you guys see any glaring bottlenecks in my layout?
>> Any other suggestions to offer (throw in more
>> controllers, different RAID layout, etc.)? Our budget
>> limit is $50k.
>
> If I had $50k budget, I'd be buying the SunFire X4500 and running
> Solaris + ZFS on it. However, you're limited to 2 dual core Opterons,
> it seems.
The HP 585 will give you quad dual core :)
Sincerely,
Joshua D. Drake
>
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-23 21:08 Vivek Khera <vivek@khera.org>
parent: Joshua D. Drake <jd@commandprompt.com>
0 siblings, 1 reply; 10+ messages in thread
From: Vivek Khera @ 2006-10-23 21:08 UTC (permalink / raw)
To: pgsql-performance
On Oct 23, 2006, at 4:59 PM, Joshua D. Drake wrote:
>> If I had $50k budget, I'd be buying the SunFire X4500 and running
>> Solaris + ZFS on it. However, you're limited to 2 dual core
>> Opterons,
>> it seems.
>
> The HP 585 will give you quad dual core :)
but can you sling the bits to and from the disk as fast as the
X4500? the speed numbers on the i/o of the x4500 are mind-numbing.
Attachments:
smime.p7s (application/pkcs7-signature, 2.5K) download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-23 21:12 Joshua D. Drake <jd@commandprompt.com>
parent: Vivek Khera <vivek@khera.org>
0 siblings, 0 replies; 10+ messages in thread
From: Joshua D. Drake @ 2006-10-23 21:12 UTC (permalink / raw)
To: Vivek Khera <vivek@khera.org>; +Cc: pgsql-performance
Vivek Khera wrote:
>
> On Oct 23, 2006, at 4:59 PM, Joshua D. Drake wrote:
>
>>> If I had $50k budget, I'd be buying the SunFire X4500 and running
>>> Solaris + ZFS on it. However, you're limited to 2 dual core Opterons,
>>> it seems.
>>
>> The HP 585 will give you quad dual core :)
>
> but can you sling the bits to and from the disk as fast as the X4500?
> the speed numbers on the i/o of the x4500 are mind-numbing.
Honestly, I don't know. I can tell you that they perform VERY well for
us (as do the 385s). I have deployed about several 385s and 585s with
the MSA30s over the last 6 months and have not been disappointed yet.
Sincerely,
Joshua D. Drake
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-23 22:08 Jim C. Nasby <jim@nasby.net>
parent: John Philips <johnphilips42@yahoo.com>
3 siblings, 0 replies; 10+ messages in thread
From: Jim C. Nasby @ 2006-10-23 22:08 UTC (permalink / raw)
To: John Philips <johnphilips42@yahoo.com>; +Cc: pgsql-performance
On Sat, Oct 21, 2006 at 08:43:05AM -0700, John Philips wrote:
> I heard some say that the transaction log should be on
> it's own array, others say it doesn't hurt to have it
> on the same array as the OS. Is it really worthwhile
> to put it on it's own array?
It all depends on the controller and how much non-WAL workload there is.
Theoretically, with a good enough controller, you can leave WAL on the
same partition as your data.
With a complex setup like you're looking at, you really will want to do
some testing to see what makes the most sense. I can also point you at a
company that does modeling of stuff like this; they could actually give
you some idea of how well that setup would perform before you buy the
hardware.
BTW, any test results you can provide back to the community would be
most appreciated!
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Optimizing disk throughput on quad Opteron
@ 2006-10-24 13:56 Bucky Jordan <bjordan@lumeta.com>
parent: John Philips <johnphilips42@yahoo.com>
0 siblings, 0 replies; 10+ messages in thread
From: Bucky Jordan @ 2006-10-24 13:56 UTC (permalink / raw)
To: John Philips <johnphilips42@yahoo.com>; Ben Suffolk <ben@vanilla.net>; +Cc: pgsql-performance
> -----Original Message-----
> From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-
> owner@postgresql.org] On Behalf Of John Philips
> Sent: Monday, October 23, 2006 8:17 AM
> To: Ben Suffolk
> Cc: pgsql-performance@postgresql.org
> Subject: Re: [PERFORM] Optimizing disk throughput on quad Opteron
>
> > The thing I would ask is would you not be better
> > with SAS drives?
> >
> > Since the comments on Dell, and the highlighted
> > issues I have been
> > looking at HP and the the Smart Array P600
> > controller with 512 BBWC.
> > Although I am looking to stick with the 8 internal
> > disks, rather than
> > use external ones.
> >
> > The HP Smart Array 50 is the external array for SAS
> > drives. Not
> > really looked into it much though.
>
> Ben,
>
> The Smart Array 50 supports a maximum of 10 disks and
> has a single I/O module, while the Smart Array 30
> supports up to 14 disks and can be configured with a
> dual I/O module.
>
> I was under the assumption that SAS runs at the same
> speed as Ultra320, in which case the Smart Array 30 is
> a better bet...
>
> Thanks for your feedback.
The drives might be about the same speed, but SAS is a completely
different bus architecture from SCSI. U320 is a parallel interface
limited to 320 MB/s for the total bus (160 MB/s per channel, so be
careful here). SAS is a 3.0Gbps direct serial interface to the drive.
So, after 5-6 drives, SAS will definitely start to pay off. Take a look
at Dell's MD1000 external enclosure vs the previous version. The MD1000
offers much better performance (not saying to go with dell, just giving
an example of SCSI vs. SAS from a vendor I'm familiar with). Oh, and if
you're not completely against dell, you can daisy chain 3 of the MD1000
enclosures together off one of their new 6850 (Quad Woodcrest) or 6950
(Quad Operton).
At the moment, the Woodcrests seem to be outperforming the Opteron in
server benchmarks, I have a quad core (dual cpu) 2950 I'd be happy to
run some pg_benches (or other preferred benchmark) if someone has a
similar opteron so we can get some relevant comparisons on the list.
Also, here's a link that was posted a while back on opteron vs.
woodcrest:
http://tweakers.net/reviews/646
HTH,
Bucky
^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2006-10-24 13:56 UTC | newest]
Thread overview: 10+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-10-21 15:43 Optimizing disk throughput on quad Opteron John Philips <johnphilips42@yahoo.com>
2006-10-21 17:33 ` Joshua D. Drake <jd@commandprompt.com>
2006-10-21 18:17 ` Ben Suffolk <ben@vanilla.net>
2006-10-23 12:16 ` John Philips <johnphilips42@yahoo.com>
2006-10-24 13:56 ` Bucky Jordan <bjordan@lumeta.com>
2006-10-23 20:56 ` Vivek Khera <vivek@khera.org>
2006-10-23 20:59 ` Joshua D. Drake <jd@commandprompt.com>
2006-10-23 21:08 ` Vivek Khera <vivek@khera.org>
2006-10-23 21:12 ` Joshua D. Drake <jd@commandprompt.com>
2006-10-23 22:08 ` Jim C. Nasby <jim@nasby.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox