pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Hardware purchase question
12+ messages / 9 participants
[nested] [flat]

* Hardware purchase question
@ 2004-11-24 17:58 Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-01 00:26 ` Re: Hardware purchase question Josh Berkus <josh@agliodbs.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  0 siblings, 2 replies; 12+ messages in thread

From: Bo Stewart @ 2004-11-24 17:58 UTC (permalink / raw)
  To: pgsql-performance

We currently are utilizing postgresql on 2 servers with the following 
configuration:

2 - 2.4 Ghz Xeon processors
4GB ram
4 36gb 10000rpm scsi drives configured for raid 10

We started out with one server and as we became IO bound we added the 
second.  We are currently considering purchasing another identical server 
to go along with these.  In addition to this we are also considering a scsi 
attached storage device in the 10 - 14 drive range configured for raid 10 
in place of the onboard 4 drives we currently have.   Daily about 30% of 
our data gets updated with about 2% new data.  Our query load is about 60% 
reads and 40% writes currently.  My question is what type of performance 
gains can I expect on average from swapping from 4 disk raid 10 to 14 disk 
raid 10?  Could I expect to see 40 - 50% better throughput.

The servers listed above are the dell 2650's which have perc 3 
controllers.  I have seen on this list where they are know for not 
performing well.  So any suggestions for an attached scsi device would be 
greatly appreciated.  Also, any thoughts on fibre channel storage devices?

Thank You,
Bo Stewart





^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
@ 2004-12-01 00:26 ` Josh Berkus <josh@agliodbs.com>
  1 sibling, 0 replies; 12+ messages in thread

From: Josh Berkus @ 2004-12-01 00:26 UTC (permalink / raw)
  To: pgsql-performance; +Cc: Bo Stewart <bstewart@marketingsolutionsinc.com>

Bo,

> 2 - 2.4 Ghz Xeon processors
> 4GB ram
> 4 36gb 10000rpm scsi drives configured for raid 10

Hopefully you've turned OFF hyperthreading?

> gains can I expect on average from swapping from 4 disk raid 10 to 14 disk
> raid 10?  Could I expect to see 40 - 50% better throughput.

This is so dependant on application design that I can't possibly estimate.  
One big gain area for you will be moving the database log (pg_xlog) to its 
own private disk resource (such as a raid-1 pair).  In high-write 
enviroments, this can gain you 15% without changing anything else.

> The servers listed above are the dell 2650's which have perc 3
> controllers.  I have seen on this list where they are know for not
> performing well.  So any suggestions for an attached scsi device would be
> greatly appreciated.  Also, any thoughts on fibre channel storage devices?

The 2650s don't perform well in a whole assortment of ways.   This is why they 
are cheap.

NetApps seem to be the current best in NAS/SAN storage, although many people 
like EMC.   Stay away from Apple's XRaid, which is not designed for 
databases.

-- 
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
@ 2004-12-13 16:33 ` Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
  2004-12-13 18:31   ` Re: Hardware purchase question Josh Berkus <josh@agliodbs.com>
  1 sibling, 2 replies; 12+ messages in thread

From: Vivek Khera @ 2004-12-13 16:33 UTC (permalink / raw)
  To: pgsql-performance

>>>>> "BS" == Bo Stewart <bstewart@marketingsolutionsinc.com> writes:

BS> The servers listed above are the dell 2650's which have perc 3
BS> controllers.  I have seen on this list where they are know for not
BS> performing well.  So any suggestions for an attached scsi device would
BS> be greatly appreciated.  Also, any thoughts on fibre channel storage
BS> devices?

I have a 2450 and a 2650 both of which are totally sucking IO wise.

The 2650 has a PERC3 card (LSI based) and has one channel holding a
mirrored pair for the pg_xlog and OS, and the other channel has 14
U320 disks in a RAID5.  If I'm lucky, I'll get 30MB/s out of the
disks.  Normally it hovers at 5 or 6MB/s on the big RAID.

I'm currently shopping for non-Dell hardware to replace it :-(

However, I keep getting conflicting advice.  My choices are along
these lines:

Dual Xeon 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
Dual Opteron 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
Dual Opteron 64bit with external RAID via fibre channel (eg, nstor)

I'm sure any of these will whip the bottom off the Dell 2650, but
which will be the fastest overall?  No way to know without spending
lots of money to test. :-(

Dell claims their new 2750 will be faster, but they've lost the battle
already, and won't commit to any performance numbers.  Won't even give
me a ballpark number.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D.                Khera Communications, Inc.
Internet: khera@kciLink.com       Rockville, MD  +1-301-869-4449 x806
AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
@ 2004-12-13 17:23   ` Joshua D. Drake <jd@commandprompt.com>
  2004-12-14 09:33     ` Re: Hardware purchase question Andrew Hood <ahood.lists@ambergreen.co.uk>
  2005-01-03 19:20     ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
  1 sibling, 2 replies; 12+ messages in thread

From: Joshua D. Drake @ 2004-12-13 17:23 UTC (permalink / raw)
  To: Vivek Khera <khera@kcilink.com>; +Cc: pgsql-performance


> However, I keep getting conflicting advice.  My choices are along
> these lines:
> 
> Dual Xeon 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with external RAID via fibre channel (eg, nstor)

An Opteron, properly tuned with PostgreSQL will always beat a Xeon
in terms of raw cpu.

RAID 10 will typically always outperform RAID 5 with the same HD config.

Fibre channel in general will always beat a normal (especially an LSI) raid.

Dell's suck for PostgreSQL.

Sincerely,

Joshua D. Drake




> 
> I'm sure any of these will whip the bottom off the Dell 2650, but
> which will be the fastest overall?  No way to know without spending
> lots of money to test. :-(
> 
> Dell claims their new 2750 will be faster, but they've lost the battle
> already, and won't commit to any performance numbers.  Won't even give
> me a ballpark number.
> 


-- 
Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

Attachments:

jd.vcf (text/x-vcard, 638B) download begin:vcard fn:Joshua D. Drake n:Drake;Joshua D. org:Command Prompt, Inc. adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA email;internet:jd@commandprompt.com title:Consultant tel;work:503-667-4564 tel;fax:503-210-0334 note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl. x-mozilla-html:FALSE url:http://www.commandprompt.com/ version:2.1 end:vcard
^ permalink raw reply [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
@ 2004-12-14 09:33     ` Andrew Hood <ahood.lists@ambergreen.co.uk>
  1 sibling, 0 replies; 12+ messages in thread

From: Andrew Hood @ 2004-12-14 09:33 UTC (permalink / raw)
  To: pgsql-performance

Joshua D. Drake wrote:
> 
> An Opteron, properly tuned with PostgreSQL will always beat a Xeon
> in terms of raw cpu.
> 
> RAID 10 will typically always outperform RAID 5 with the same HD config.
> 
> Fibre channel in general will always beat a normal (especially an LSI) 
> raid.
> 
> Dell's suck for PostgreSQL.

Does anyone have any OS recommendations/experiences for PostgreSQL on 
Opteron?

Thanks,
Andrew



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
@ 2005-01-03 19:20     ` Mitch Pirtle <mitch.pirtle@gmail.com>
  2005-01-03 19:35       ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
  1 sibling, 1 reply; 12+ messages in thread

From: Mitch Pirtle @ 2005-01-03 19:20 UTC (permalink / raw)
  To: pgsql-performance

On Mon, 13 Dec 2004 09:23:13 -0800, Joshua D. Drake
<jd@commandprompt.com> wrote:
> 
> RAID 10 will typically always outperform RAID 5 with the same HD config.

Isn't RAID10 just RAID5 mirrored?  How does that speed up performance?
 Or am I missing something?

-- Mitch



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
  2005-01-03 19:20     ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
@ 2005-01-03 19:35       ` Madison Kelly <linux@alteeve.com>
  2005-01-03 20:19         ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
  2005-01-03 20:36         ` Re: Hardware purchase question Greg Stark <gsstark@mit.edu>
  0 siblings, 2 replies; 12+ messages in thread

From: Madison Kelly @ 2005-01-03 19:35 UTC (permalink / raw)
  To: pgsql-performance

Mitch Pirtle wrote:
> On Mon, 13 Dec 2004 09:23:13 -0800, Joshua D. Drake
> <jd@commandprompt.com> wrote:
> 
>>RAID 10 will typically always outperform RAID 5 with the same HD config.
> 
> 
> Isn't RAID10 just RAID5 mirrored?  How does that speed up performance?
>  Or am I missing something?
> 
> -- Mitch

Hi Mitch,

   Nope, Raid 10 (one zero) is a mirror is stripes, no parity. with r10 
you get the benefit of a full mirror which means your system does not 
need to calculate the XOR parity but you only get 50% disk usage. The 
mirror causes a slight write hit as the data needs to be split between 
two disk (or in this case, to striped pairs) but reads can be up to 
twice as fast (theoretically). By adding the stripe you negate the write 
hit and actually gain write performance because half the data goes to 
mirror A, half to mirror B (same with reads, roughly).

   Raid 10 is a popular choice for software raid because of the reduced 
overhead. Raid 5 on the otherhand does require that a parity bit is 
calculated for every N-1 disks. With r5 you get N-1 disk usage (you get 
the combined capacity of 3 disks in a 4 disk r5 array) and still get the 
benefit of striping across the disks so long as you have a dedicated 
raid asic that can do the XOR calculations. Without it, specially in a 
failure state, the performance can collapse as the CPU performs all that 
extra math.

hth

Madison



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
  2005-01-03 19:20     ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
  2005-01-03 19:35       ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
@ 2005-01-03 20:19         ` Madison Kelly <linux@alteeve.com>
  2005-01-03 20:44           ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
  1 sibling, 1 reply; 12+ messages in thread

From: Madison Kelly @ 2005-01-03 20:19 UTC (permalink / raw)
  To: pgsql-performance

Madison Kelly wrote:
>   Nope, Raid 10 (one zero) is a mirror is stripes, no parity. with r10 

Woops, that should be "mirror of stripes".

By the way, what you are thinking of is possible, it would be 51 (five 
one; a raid 5 built on mirrors) or 15 (a mirror of raid 5 arrays). 
Always be careful, 10 and 01 are also not the same. You want to think 
carefully about what you want out of your array before building it.

Madison



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
  2005-01-03 19:20     ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
  2005-01-03 19:35       ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
  2005-01-03 20:19         ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
@ 2005-01-03 20:44           ` Mitch Pirtle <mitch.pirtle@gmail.com>
  2005-01-04 01:21             ` Re: Hardware purchase question Grega Bremec <gregab@p0f.net>
  0 siblings, 1 reply; 12+ messages in thread

From: Mitch Pirtle @ 2005-01-03 20:44 UTC (permalink / raw)
  To: Madison Kelly <linux@alteeve.com>; +Cc: pgsql-performance

You are right, I now remember that setup was originally called "RAID
10 plus 1", and I believe is was an incorrect statement from an
overzealous salesman ;-)

Thanks for the clarification!

- Mitch

On Mon, 03 Jan 2005 15:19:04 -0500, Madison Kelly <linux@alteeve.com> wrote:
> Madison Kelly wrote:
> >   Nope, Raid 10 (one zero) is a mirror is stripes, no parity. with r10
> 
> Woops, that should be "mirror of stripes".
> 
> By the way, what you are thinking of is possible, it would be 51 (five
> one; a raid 5 built on mirrors) or 15 (a mirror of raid 5 arrays).
> Always be careful, 10 and 01 are also not the same. You want to think
> carefully about what you want out of your array before building it.



^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
  2005-01-03 19:20     ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
  2005-01-03 19:35       ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
  2005-01-03 20:19         ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
  2005-01-03 20:44           ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
@ 2005-01-04 01:21             ` Grega Bremec <gregab@p0f.net>
  0 siblings, 0 replies; 12+ messages in thread

From: Grega Bremec @ 2005-01-04 01:21 UTC (permalink / raw)
  To: Mitch Pirtle <mitch.pirtle@gmail.com>; +Cc: pgsql-performance

...and on Mon, Jan 03, 2005 at 03:44:44PM -0500, Mitch Pirtle used the keyboard:
>
> You are right, I now remember that setup was originally called "RAID
> 10 plus 1", and I believe is was an incorrect statement from an
> overzealous salesman ;-)
>

Just an afterthought - that could well be the unfortunate consequence of
salesmen specializing in sales as an act rather than the goods they were
selling - it might be that he/she was referring to the specifics of the
concrete configuration they were selling you (or trying to sell you),
which should, in the case you were mentioning, probably be called "a
RAID10 array with a hotspare drive" - that is, it would be preconfigured
to, upon the failure of one of array members, detect the failed drive and
automatically replace it with one that has been sitting there all the time,
doing nothing but waiting for one of its active companions to fail.

But this already falls into the category that has, so far, probably
caused the vast majority of misunderstandings, failed investments and
grey hair in RAID, namely data safety, and I don't feel particularly
qualified for getting into specifics of this at this moment, as it
happens to be 2AM, I had a couple of beers (my friend's birthday's due)
and I'm dying to get some sleep. :)

HTH, cheers,
-- 
    Grega Bremec
    gregab at p0f dot net

^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
  2004-12-13 17:23   ` Re: Hardware purchase question Joshua D. Drake <jd@commandprompt.com>
  2005-01-03 19:20     ` Re: Hardware purchase question Mitch Pirtle <mitch.pirtle@gmail.com>
  2005-01-03 19:35       ` Re: Hardware purchase question Madison Kelly <linux@alteeve.com>
@ 2005-01-03 20:36         ` Greg Stark <gsstark@mit.edu>
  1 sibling, 0 replies; 12+ messages in thread

From: Greg Stark @ 2005-01-03 20:36 UTC (permalink / raw)
  To: Madison Kelly <linux@alteeve.com>; +Cc: pgsql-performance


Madison Kelly <linux@alteeve.com> writes:

> Without it, specially in a failure state, the performance can collapse as
> the CPU performs all that extra math.

It's really not the math that makes raid 5 hurt. It's that in order to
calculate the checksum block the raid controller needs to read in the existing
checksum block and write out the new version. So every write causes not just
one drive seeking and writing, but a second drive seeking and performing a
read and a write.

The usual strategy for dealing with that is stuffing a huge nonvolatile cache
in the controller so those reads are mostly cached and the extra writes don't
saturate the i/o throughput. But those kinds of controllers are expensive and
not an option for software raid.

-- 
greg




^ permalink  raw  reply  [nested|flat] 12+ messages in thread

* Re: Hardware purchase question
  2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
  2004-12-13 16:33 ` Re: Hardware purchase question Vivek Khera <khera@kcilink.com>
@ 2004-12-13 18:31   ` Josh Berkus <josh@agliodbs.com>
  1 sibling, 0 replies; 12+ messages in thread

From: Josh Berkus @ 2004-12-13 18:31 UTC (permalink / raw)
  To: pgsql-performance; +Cc: Vivek Khera <khera@kcilink.com>

Vivek,

> Dual Xeon 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with external RAID via fibre channel (eg, nstor)

Opteron over Xeon, no question.    Not only are the Opterons 
real-world-faster, they are less severely affected by the CS bug.

> I'm sure any of these will whip the bottom off the Dell 2650, but
> which will be the fastest overall?  No way to know without spending
> lots of money to test. :-(

The SAN is going to be faster with a good SAN.   That being said, I understand 
that "a good SAN" is something like a $30,000 NetApp; the less expensive 
SANs/NASes don't seem to be more than an external drive enclosure with a raid 
chip (e.g. Apple XRaid).   But we saw even a less expensive/slower EMC 
machine improve performance just moving the pg_xlog off of the local PERC 
RAID 5 and onto the SAN.  So this is probably a good way to go if you can 
afford it.

-- 
Josh Berkus
Aglio Database Solutions
San Francisco



^ permalink  raw  reply  [nested|flat] 12+ messages in thread


end of thread, other threads:[~2005-01-04 01:21 UTC | newest]

Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-11-24 17:58 Hardware purchase question Bo Stewart <bstewart@marketingsolutionsinc.com>
2004-12-01 00:26 ` Josh Berkus <josh@agliodbs.com>
2004-12-13 16:33 ` Vivek Khera <khera@kcilink.com>
2004-12-13 17:23   ` Joshua D. Drake <jd@commandprompt.com>
2004-12-14 09:33     ` Andrew Hood <ahood.lists@ambergreen.co.uk>
2005-01-03 19:20     ` Mitch Pirtle <mitch.pirtle@gmail.com>
2005-01-03 19:35       ` Madison Kelly <linux@alteeve.com>
2005-01-03 20:19         ` Madison Kelly <linux@alteeve.com>
2005-01-03 20:44           ` Mitch Pirtle <mitch.pirtle@gmail.com>
2005-01-04 01:21             ` Grega Bremec <gregab@p0f.net>
2005-01-03 20:36         ` Greg Stark <gsstark@mit.edu>
2004-12-13 18:31   ` Josh Berkus <josh@agliodbs.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox