pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How to Improve w/7K$?
6+ messages / 6 participants
[nested] [flat]

* Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How to Improve w/7K$?
@ 2005-04-15 16:17 Mohan, Ross <RMohan@arbinet.com>
  2005-04-15 18:11 ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Steve Poe <spoe@sfnet.cc>
  0 siblings, 1 reply; 6+ messages in thread

From: Mohan, Ross @ 2005-04-15 16:17 UTC (permalink / raw)
  To: pgsql-performance


Sorry to blend threads, but in my kinda longish, somewhat thankless, 
essentially anonymous, and quite average career as a dba, I have 
found that the 7K would be best spent on a definitive end-to-end
"application critical path" test (pretty easy to instrument apps
and lash on test harnesses these days). 


If it's "the disk subsystem", then by all means, spend the 7K there. 

If the "7K$" is for "hardware only", then disk is always a good choice. For
a really small shop, maybe it's an upgrade to a dual CPU opteron MOBO, eg. 
dunno.

If, however, in the far-more-likely case that the application code
or system/business process is the throttle point, it'd be a great
use of money to have a test report showing that to the "higher ups". 
That's where the best scalability bang-for-buck can be made. 


- Ross

p.s. having said this, and as already been noted "7K" ain't
     going to buy that much....maybe the ability to go RAID 10?
     
p.p.s  Why don't we start a PGSQL-7K listserv, to handle this EPIC thread? :-)


-----Original Message-----
From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Richard_D_Levine@raytheon.com
Sent: Friday, April 15, 2005 11:20 AM
To: Alex Turner
Cc: Dave Held; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org
Subject: Re: [PERFORM] Intel SRCS16 SATA raid?


This is a different thread that the $7k server thread.
Greg Stark started it and wrote:
                                                                    
 "I'm also wondering about whether I'm better off with one of these 
 SATA raid                                                          
 controllers or just going with SCSI drives."                       
                                                                    
                                                                    

Rick

pgsql-performance-owner@postgresql.org wrote on 04/15/2005 10:01:56 AM:

> The original thread was how much can I get for $7k
>
> You can't fit a 15k RPM SCSI solution into $7K ;)  Some of us are ona
budget!
>
> 10k RPM SATA drives give acceptable performance at a good price, thats 
> really the point here.
>
> I have never really argued that SATA is going to match SCSI 
> performance on multidrive arrays for IO/sec.  But it's all about the 
> benjamins baby.  If I told my boss we need $25k for a database 
> machine, he'd tell me that was impossible, and I have $5k to do it. If 
> I tell him $7k - he will swallow that.  We don't _need_ the amazing 
> performance of a 15k RPM drive config.  Our biggest hit is reads, so 
> we can buy 3xSATA machines and load balance.  It's all about the 
> application, and buying what is appropriate.  I don't buy a Corvette 
> if all I need is a malibu.
>
> Alex Turner
> netEconomist
>
> On 4/15/05, Dave Held <dave.held@arrayservicesgrp.com> wrote:
> > > -----Original Message-----
> > > From: Alex Turner [mailto:armtuk@gmail.com]
> > > Sent: Thursday, April 14, 2005 6:15 PM
> > > To: Dave Held
> > > Cc: pgsql-performance@postgresql.org
> > > Subject: Re: [PERFORM] Intel SRCS16 SATA raid?
> > >
> > > Looking at the numbers, the raptor with TCQ enabled was close or 
> > > beat the Atlas III 10k drive on most benchmarks.
> >
> > And I would be willing to bet that the Atlas 10k is not using the 
> > same generation of technology as the Raptors.
> >
> > > Naturaly a 15k drive is going to be faster in many areas, but it 
> > > is also much more expensive.  It was only 44% better on the server 
> > > tests than the raptor with TCQ, but it costs nearly 300% more 
> > > ($538 cdw.com, $180 newegg.com).
> >
> > State that in terms of cars.  Would you be willing to pay 300% more 
> > for a car that is 44% faster than your competitor's?  Of course you 
> > would, because we all recognize that the cost of speed/performance 
> > does not scale linearly.  Naturally, you buy the best speed that you 
> > can afford, but when it comes to hard drives, the only major feature 
> > whose price tends to scale anywhere close to linearly is capacity.
> >
> > > Note also that the 15k drive was the only drive that kept up with 
> > > the raptor on raw transfer speed, which is going to matter for 
> > > WAL.
> >
> > So get a Raptor for your WAL partition. ;)
> >
> > > [...]
> > > The Raptor drives can be had for as little as $180/ea, which is 
> > > quite a good price point considering they can keep up with their 
> > > SCSI 10k RPM counterparts on almost all tests with NCQ enabled 
> > > (Note that 3ware controllers _don't_ support NCQ, although they 
> > > claim their HBA based queueing is 95% as good as NCQ on the 
> > > drive).
> >
> > Just keep in mind the points made by the Seagate article.  You're 
> > buying much more than just performance for that $500+.  You're also 
> > buying vibrational tolerance, high MTBF, better internal 
> > environmental controls, and a pretty significant margin on seek 
> > time, which is probably your most important feature for disks 
> > storing tables. An interesting test would be to stick several drives 
> > in a cabinet and graph how performance is affected at the different 
> > price points/ technologies/number of drives.
> >
> > __
> > David B. Held
> > Software Engineer/Array Services Group
> > 200 14th Ave. East,  Sartell, MN 56377
> > 320.534.3637 320.253.7800 800.752.8129
> >
> > ---------------------------(end of
broadcast)---------------------------
> > TIP 3: if posting/reading through Usenet, please send an appropriate
> >       subscribe-nomail command to majordomo@postgresql.org so that your
> >       message can get through to the mailing list cleanly
> >
>
> ---------------------------(end of 
> broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

               http://www.postgresql.org/docs/faq



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How
  2005-04-15 16:17 Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How to Improve w/7K$? Mohan, Ross <RMohan@arbinet.com>
@ 2005-04-15 18:11 ` Steve Poe <spoe@sfnet.cc>
  0 siblings, 0 replies; 6+ messages in thread

From: Steve Poe @ 2005-04-15 18:11 UTC (permalink / raw)
  To: Mohan, Ross <RMohan@arbinet.com>; +Cc: pgsql-performance

Ross,

I agree with you, but I' am the lowly intergrator/analyst, I have to solve the problem without all
the authority (sounds like a project manager). I originally started this thread since I had the $7k budget.

I am not a dba/developer. but I play one on t.v., so I can only assume that throwing money
at the application code means one understand what the bottleneck in the code and what it takes to fix it.

In this situation, the code is hidden by the vendor that connects to the database. So, besides persisent requests of the vendor to improve the area of the application, the balance of tuning lies with the hardware. The answer is *both* hardware and application code. Finding the right balance is key. Your mileage may vary.

Steve Poe

 


>If, however, in the far-more-likely case that the application code
>or system/business process is the throttle point, it'd be a great
>use of money to have a test report showing that to the "higher ups". 
>That's where the best scalability bang-for-buck can be made. 


- Ross

>p.s. having said this, and as already been noted "7K" ain't
>     going to buy that much....maybe the ability to go RAID 10?
     
>p.p.s  Why don't we start a PGSQL-7K listserv, to handle this EPIC thread?  :-) 





^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How
@ 2005-04-15 20:43 Matthew Nuzum <matt.followers@gmail.com>
  2005-04-15 21:33 ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Rod Taylor <pg@rbt.ca>
  0 siblings, 1 reply; 6+ messages in thread

From: Matthew Nuzum @ 2005-04-15 20:43 UTC (permalink / raw)
  To: pgsql-performance; 'Steve Poe' <spoe@sfnet.cc>

I think there are many people who feel that $7,000 is a good budget for a
database server, me being one.

 * I agree with the threads that more disks are better.
 * I also agree that SCSI is better, but can be hard to justify if your
budget is tight, and I have great certainty that 2x SATA drives on a good
controller is better than x SCSI drives for many work loads.
 * I also feel that good database design and proper maintenance can be one
of the single biggest performance enhancers available. This can be labor
intensive, however, and sometimes throwing more hardware at a problem is
cheaper than restructuring a db.

Either way, having a good hardware platform is an excellent place to start,
as much of your tuning will depend on certain aspects of your hardware.

So if you need a db server, and you have $7k to spend, I'd say spend it.
From this list, I've gathered that I/O and RAM are your two most important
investments.

Once you get that figured out, you can still do some performance tuning on
your new server using the excellent advice from this mailing list.

By the way, for all those who make this list work, I've rarely found such a
thorough, helpful and considerate group of people as these on the
performance list.

-- 
Matthew Nuzum <matt@followers.net>
www.followers.net - Makers of "Elite Content Management System"
View samples of Elite CMS in action by visiting
http://www.followers.net/portfolio/






^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How
  2005-04-15 20:43 Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Matthew Nuzum <matt.followers@gmail.com>
@ 2005-04-15 21:33 ` Rod Taylor <pg@rbt.ca>
  2005-04-16 00:10   ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Ron Mayer <rm_pg@cheapcomplexdevices.com>
  0 siblings, 1 reply; 6+ messages in thread

From: Rod Taylor @ 2005-04-15 21:33 UTC (permalink / raw)
  To: matt@followers.net; +Cc: Steve Poe <spoe@sfnet.cc>; pgsql-performance

On Fri, 2005-04-15 at 15:43 -0500, Matthew Nuzum wrote:
> I think there are many people who feel that $7,000 is a good budget for a
> database server, me being one.

The budget for a database server is usually some %age of the value of
the data within the database or the value of it's availability. Big
budget hardware (well, going from $7k to $100k) often brings more
redundancy and reliability improvement than performance improvement.

If you're going to lose $100k in business because the database was
unavailable for 12 hours, then kick $75k into the hardware and call a
profit of $25k over 3 years (hardware lifetime is 3 years, catastrophic
failure happens once every 3 or so years...).

Ditto for backup systems. If the company depends on the data in the
database for it's survival, where bankruptcy or worse would happen as a
result of complete dataloss, then it would be a good idea to invest a
significant amount of the companies revenue into making damn sure that
doesn't happen. Call it an insurance policy.


Performance for me dictates which hardware is purchased and
configuration is used within $BUDGET, but $BUDGET itself is nearly
always defined by the value of the data stored.


>  * I agree with the threads that more disks are better.
>  * I also agree that SCSI is better, but can be hard to justify if your
> budget is tight, and I have great certainty that 2x SATA drives on a good
> controller is better than x SCSI drives for many work loads.
>  * I also feel that good database design and proper maintenance can be one
> of the single biggest performance enhancers available. This can be labor
> intensive, however, and sometimes throwing more hardware at a problem is
> cheaper than restructuring a db.
> 
> Either way, having a good hardware platform is an excellent place to start,
> as much of your tuning will depend on certain aspects of your hardware.
> 
> So if you need a db server, and you have $7k to spend, I'd say spend it.
> >From this list, I've gathered that I/O and RAM are your two most important
> investments.
> 
> Once you get that figured out, you can still do some performance tuning on
> your new server using the excellent advice from this mailing list.
> 
> By the way, for all those who make this list work, I've rarely found such a
> thorough, helpful and considerate group of people as these on the
> performance list.
> 
-- 




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How
  2005-04-15 20:43 Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Matthew Nuzum <matt.followers@gmail.com>
  2005-04-15 21:33 ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Rod Taylor <pg@rbt.ca>
@ 2005-04-16 00:10   ` Ron Mayer <rm_pg@cheapcomplexdevices.com>
  2005-04-20 15:24     ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Vivek Khera <vivek@khera.org>
  0 siblings, 1 reply; 6+ messages in thread

From: Ron Mayer @ 2005-04-16 00:10 UTC (permalink / raw)
  To: Rod Taylor <pg@rbt.ca>

Rod Taylor wrote:
 > On Fri, 2005-04-15 at 15:43 -0500, Matthew Nuzum wrote:
 >> * I agree with the threads that more disks are better.
 >> * I also agree that SCSI is better, but can be hard to justify

Here's another approach to spend $7000 that we're currently
trying.... but it'll only work for certain systems if you can
use load balancing and/or application level partitioning
of your software.

For $859 you can buy
     a Dell SC1425 with  (*see footnote)
     2 Xeon 2.8GHz processors  (*see footnote)
     1 GB ram
     1 80GB hard drive. (*see footnote)

Doing the math, it seems I could get 8 of
these systems for that $6870, giving me:
    16  Xeon processors (*see footnote),
    640 GB of disk space spread over 8 spindles
    8   GB of ram
    16  1Gbps network adapters.


Despite the non-optimal hardware (* see footnote), the price
of each system and extra redundancy may make up the difference
for some applications.

For example, I didn't see many other $7000 proposals have
have nearly 10GB of ram, or over a dozen CPUs (even counting
the raid controllers), or over a half a terrabyte of storage ,
or capable of 5-10 Gbit/sec of network traffic...  The extra
capacity would allow me to have redundancy that would somewhat
make up for the flakier hardware, no raid, etc.

Thoughts?  Over the next couple months I'll be evaluating
a cluster of 4 systems almost exactly as I described (but
with cheaper dual hard drives in each system), for a GIS
system that does lend itself well to application-level
partitioning.

     Ron

(* footnotes)
  Yeah, I know some reports here say that dual Xeons can suck;
  but Dell's throwing in the second one for free.
  Yeah, I know some reports here say Dells can suck, but it
  was easy to get a price quote online, and they're a nice
  business partner of ours.
  Yeah, I should get 2 hard drives in each system, but Dell
  wanting an additional $160 for a 80GB hard drive is not a good deal.
  Yeah, I know I'd be better off with 2GB ram, but Dell
  wants $400 (half the price of an entire additional
  system) for the upgrade from 1GB to 2.

  I also realize that application level partitioning needed
  to take advantage of a loose cluster like this is not practical
  for many applications.



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How
  2005-04-15 20:43 Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Matthew Nuzum <matt.followers@gmail.com>
  2005-04-15 21:33 ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Rod Taylor <pg@rbt.ca>
  2005-04-16 00:10   ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Ron Mayer <rm_pg@cheapcomplexdevices.com>
@ 2005-04-20 15:24     ` Vivek Khera <vivek@khera.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Vivek Khera @ 2005-04-20 15:24 UTC (permalink / raw)
  To: pgsql-performance


On Apr 15, 2005, at 8:10 PM, Ron Mayer wrote:

> For example, I didn't see many other $7000 proposals have
> have nearly 10GB of ram, or over a dozen CPUs (even counting
> the raid controllers), or over a half a terrabyte of storage ,
> or capable of 5-10 Gbit/sec of network traffic...  The extra

And how much are you spending on the switch that will carry 10Gb/sec 
traffic?

> capacity would allow me to have redundancy that would somewhat
> make up for the flakier hardware, no raid, etc.

it would work for some class of applications which are pretty much 
read-only.  and don't forget to factor in the overhead of the 
replication...

>
> Thoughts?  Over the next couple months I'll be evaluating
> a cluster of 4 systems almost exactly as I described (but
> with cheaper dual hard drives in each system), for a GIS
> system that does lend itself well to application-level
> partitioning.

I'd go with fewer bigger boxes with RAID so i can sleep better at night 
:-)


Vivek Khera, Ph.D.
+1-301-869-4449 x806

Attachments:

  smime.p7s (application/pkcs7-signature, 2.4K) download

^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2005-04-20 15:24 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-04-15 16:17 Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How to Improve w/7K$? Mohan, Ross <RMohan@arbinet.com>
2005-04-15 18:11 ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Steve Poe <spoe@sfnet.cc>
2005-04-15 20:43 Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Matthew Nuzum <matt.followers@gmail.com>
2005-04-15 21:33 ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Rod Taylor <pg@rbt.ca>
2005-04-16 00:10   ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Ron Mayer <rm_pg@cheapcomplexdevices.com>
2005-04-20 15:24     ` Re: Spend 7K *WHERE*? WAS Intel SRCS16 SATA raid? and How Vivek Khera <vivek@khera.org>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox