pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
software vs hw hard on linux
7+ messages / 6 participants
[nested] [flat]

* software vs hw hard on linux
@ 2003-09-12 14:34 Jeff <threshar@torgo.978.org>
  2003-09-12 14:49 ` Re: software vs hw hard on linux aturner@neteconomist.com
  2003-09-12 15:32 ` Re: software vs hw hard on linux Christopher Browne <cbbrowne@libertyrms.info>
  2003-09-12 16:55 ` Re: software vs hw hard on linux Josh Berkus <josh@agliodbs.com>
  2003-09-12 19:04 ` Re: software vs hw hard on linux Vivek Khera <khera@kcilink.com>
  2003-09-14 05:40 ` Re: software vs hw hard on linux Cott Lang <cott@internetstaff.com>
  0 siblings, 5 replies; 7+ messages in thread

From: Jeff @ 2003-09-12 14:34 UTC (permalink / raw)
  To: pgsql-performance

Due to various third party issues, and the fact PG rules, we're planning
on migrating our deplorable informix db to PG.  It is a rather large DB
with a rather high amount of activity (mostly updates).  So I'm going to
be aquiring a dual (or quad if they'll give me money) box. (In my testing
my glorious P2 with a 2 spindle raid0 is able to handle it fairly well)

What I'm wondering about is what folks experience with software raid vs
hardware raid on linux is.  A friend of mine ran a set of benchmarks at
work and found sw raid was running obscenely faster than the mylex and
(some other brand that isn't 3ware) raids..

On the pro-hw side you have ones with battery backed cache, chacnes are
they are less likely to fail..

On the pro-sw side you have lots of speed and less cost (unfortunately,
there is a pathetic budget so spending $15k on a raid card is out of the
question really).

any thoughts?

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/





^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: software vs hw hard on linux
  2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
@ 2003-09-12 14:49 ` aturner@neteconomist.com
  2003-09-12 19:03   ` Re: software vs hw hard on linux Vivek Khera <khera@kcilink.com>
  4 siblings, 1 reply; 7+ messages in thread

From: aturner@neteconomist.com @ 2003-09-12 14:49 UTC (permalink / raw)
  To: Jeff <threshar@torgo.978.org>; +Cc: pgsql-performance

My personal experience with RAID cards is that you have to spend money to get good performance.  You need battery backed cache because RAID 5 only works well with write to cache turned on, and you need a good size cache too.  If you don't have it, RAID 5 performance will suck big time.  If you need speed, RAID 10 seems to be the only way to go, but of course that means you are gonna spend $$s on drives and chasis.  I wish someone would start a website like storagereview.com for RAID cards because I have had _vastly_ differing experience with different cards.  We currently have a compaq ML370 with a Compaq Smart Array 5300, and quite frankly it sucks (8MB/sec write).  I get better performance numbers off my new Tyan Thunder s2469UGN board with a single U320 10k RPM drive (50MB/sec) than we get off our RAID 5 array including seeks/sec.  Definately shop around, and hopefully some other folks can give some suggestions of a good RAID card, and a good config.

Alex Turner

P.S. If there is movement for a RAID review site, I would be willing to start one, I'm pretty dissapointed at the lack of resources out there for this.

On Fri, Sep 12, 2003 at 10:34:26AM -0400, Jeff wrote:
> Due to various third party issues, and the fact PG rules, we're planning
> on migrating our deplorable informix db to PG.  It is a rather large DB
> with a rather high amount of activity (mostly updates).  So I'm going to
> be aquiring a dual (or quad if they'll give me money) box. (In my testing
> my glorious P2 with a 2 spindle raid0 is able to handle it fairly well)
> 
> What I'm wondering about is what folks experience with software raid vs
> hardware raid on linux is.  A friend of mine ran a set of benchmarks at
> work and found sw raid was running obscenely faster than the mylex and
> (some other brand that isn't 3ware) raids..
> 
> On the pro-hw side you have ones with battery backed cache, chacnes are
> they are less likely to fail..
> 
> On the pro-sw side you have lots of speed and less cost (unfortunately,
> there is a pathetic budget so spending $15k on a raid card is out of the
> question really).
> 
> any thoughts?
> 
> --
> Jeff Trout <jeff@jefftrout.com>
> http://www.jefftrout.com/
> http://www.stuarthamm.net/
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: software vs hw hard on linux
  2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
  2003-09-12 14:49 ` Re: software vs hw hard on linux aturner@neteconomist.com
@ 2003-09-12 19:03   ` Vivek Khera <khera@kcilink.com>
  0 siblings, 0 replies; 7+ messages in thread

From: Vivek Khera @ 2003-09-12 19:03 UTC (permalink / raw)
  To: pgsql-performance

>>>>> "a" == aturner  <aturner@neteconomist.com> writes:

a> you need a good size cache too.  If you don't have it, RAID 5
a> performance will suck big time.  If you need speed, RAID 10 seems
a> to be the only way to go, but of course that means you are gonna
a> spend $$s on drives and chasis.  I wish someone would start a

I disagree on your RAID level assertions.  Check back about 10 or 15
days on this list for some numbers I posted on restore times for a 20+
GB database with different RAID levels.  RAID5 came out fastest
compared with RAID10 and RAID50 across 14 disks.  On my 5 disk system,
I run RAID10 plus a spare in preference to RAID5 as it is faster for
that.  So the answer is "it depends". ;-)

Both systems use SCSI hardware RAID controllers, one is LSI and the
other Adaptec, all hardware from Dell.

But if you're budget limited, spend every last penny you have on the
fastest disks you can get, and then boost memory.  Any current CPU
will be more than enough for Postgres.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D.                Khera Communications, Inc.
Internet: khera@kciLink.com       Rockville, MD       +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: software vs hw hard on linux
  2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
@ 2003-09-12 15:32 ` Christopher Browne <cbbrowne@libertyrms.info>
  4 siblings, 0 replies; 7+ messages in thread

From: Christopher Browne @ 2003-09-12 15:32 UTC (permalink / raw)
  To: pgsql-performance

threshar@torgo.978.org (Jeff) writes:
> On the pro-sw side you have lots of speed and less cost (unfortunately,
> there is a pathetic budget so spending $15k on a raid card is out of the
> question really).

I have been playing with a Perq3 QC card
  <http://www.scsi4me.com/?menu=menu_scsi&pid=143;
which isn't anywhere near $15K, and which certainly seems to provide the
characteristic improved performance.

PriceWatch is showing several LSI Logic cards in the $300-$400 range
with battery backed cache, which doesn't seem too out of line.  

It would seem a good tradeoff to buy one of these cards and drop a
SCSI drive off the array.
-- 
output = ("cbbrowne" "@" "libertyrms.info")
<http://dev6.int.libertyrms.com/;
Christopher Browne
(416) 646 3304 x124 (land)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: software vs hw hard on linux
  2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
@ 2003-09-12 16:55 ` Josh Berkus <josh@agliodbs.com>
  4 siblings, 0 replies; 7+ messages in thread

From: Josh Berkus @ 2003-09-12 16:55 UTC (permalink / raw)
  To: Jeff <threshar@torgo.978.org>; pgsql-performance

Jeff,

> What I'm wondering about is what folks experience with software raid vs
> hardware raid on linux is.  A friend of mine ran a set of benchmarks at
> work and found sw raid was running obscenely faster than the mylex and
> (some other brand that isn't 3ware) raids..

Our company has stopped recommending hardware raid for all low-to-medium end 
systems.   Our experience is that Linux SW RAID does as good a job as any 
$700 to $1000 RAID card, and has the advantage of not having lots of driver 
issues (for example, we still have one system running Linux 2.2.19 because 
the Mylex driver maintainer passed away in early 2002).

The exception to this is if you are expecting to frequently max out your CPU 
and/or RAM with your application, in which case the SW RAID might not be so 
good because you would get query-vs.-RAID CPU contention.

-- 
-Josh Berkus
 Aglio Database Solutions
 San Francisco




^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: software vs hw hard on linux
  2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
@ 2003-09-12 19:04 ` Vivek Khera <khera@kcilink.com>
  4 siblings, 0 replies; 7+ messages in thread

From: Vivek Khera @ 2003-09-12 19:04 UTC (permalink / raw)
  To: pgsql-performance

>>>>> "J" == Jeff  <threshar@torgo.978.org> writes:

J> Due to various third party issues, and the fact PG rules, we're planning
J> on migrating our deplorable informix db to PG.  It is a rather large DB
J> with a rather high amount of activity (mostly updates).  So I'm going to

If at all possible, batch your updates within transactions containing
as many of those updates as you can.  You will get *much* better
performance.

More than 2 procs is probably overkill.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D.                Khera Communications, Inc.
Internet: khera@kciLink.com       Rockville, MD       +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: software vs hw hard on linux
  2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
@ 2003-09-14 05:40 ` Cott Lang <cott@internetstaff.com>
  4 siblings, 0 replies; 7+ messages in thread

From: Cott Lang @ 2003-09-14 05:40 UTC (permalink / raw)
  To: pgsql-performance

On Fri, 2003-09-12 at 07:34, Jeff wrote:

> What I'm wondering about is what folks experience with software raid vs
> hardware raid on linux is.  A friend of mine ran a set of benchmarks at
> work and found sw raid was running obscenely faster than the mylex and
> (some other brand that isn't 3ware) raids..

I ended up going with a hybrid:  RAID-1 across sets of two disks in
hardware on Adaptec ZCR cards, and RAID-0 across the RAID-1s with Linux
software RAID.

Although the ZCR (2010 I believe) supports 0+1, using software striping
turned in better performance for me.

This way, I get brain dead simple dead disk replacement handled by
hardware with some speed from software RAID.

Also, I would think mirroring on the SCSI controller should take traffic
off the PCI bus... <shrug>

I have another machine that's stuck using a Compaq 5i plus controller
with no battery backed write cache, in RAID 5. It sucks. Really bad. I'd
rather use an IDE drive. :)








^ permalink  raw  reply  [nested|flat] 7+ messages in thread


end of thread, other threads:[~2003-09-14 05:40 UTC | newest]

Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-09-12 14:34 software vs hw hard on linux Jeff <threshar@torgo.978.org>
2003-09-12 14:49 ` aturner@neteconomist.com
2003-09-12 19:03   ` Vivek Khera <khera@kcilink.com>
2003-09-12 15:32 ` Christopher Browne <cbbrowne@libertyrms.info>
2003-09-12 16:55 ` Josh Berkus <josh@agliodbs.com>
2003-09-12 19:04 ` Vivek Khera <khera@kcilink.com>
2003-09-14 05:40 ` Cott Lang <cott@internetstaff.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox