pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Follow-Up: How to improve db performance with $7K?
9+ messages / 7 participants
[nested] [flat]

* Follow-Up: How to improve db performance with $7K?
@ 2005-04-01 02:01 Steve Poe <spoe@sfnet.cc>
  2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
  2005-04-01 21:23 ` Re: Follow-Up: How to improve db performance with $7K? Vivek Khera <vivek@khera.org>
  0 siblings, 2 replies; 9+ messages in thread

From: Steve Poe @ 2005-04-01 02:01 UTC (permalink / raw)
  To: ; +Cc: pgsql-performance

Thanks for everyone's feedback on to best improve our Postgresql 
database for the animal hospital. I re-read the PostgreSQL 8.0 
Performance Checklist just to keep focused.

We purchased (2) 4 x 146GB 10,000rpm SCSI U320 SCA drive arrays ($2600) 
and (1) Sun W2100z dual AMD64 workstation with 4GB RAM ($2500). We did 
not need a rack-mount server, so I though Sun's workstation would do 
fine. I'll double the RAM. Hopefully, this should out-perform our dual 
2.8 Xeon with 4GB of RAM.

Now, we need to purchase a good U320 RAID card now. Any suggestions for 
those  which run well under Linux?

These two drive arrays main purpose is for our database. For those 
messed with drive arrays before, how would you slice-up the drive array? 
Will database performance be effected how our RAID10 is configured? Any 
suggestions?

Thanks.

Steve Poe






^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
@ 2005-04-01 08:09 ` Thomas F.O'Connell <tfo@sitening.com>
  2005-04-06 04:44   ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
  1 sibling, 1 reply; 9+ messages in thread

From: Thomas F.O'Connell @ 2005-04-01 08:09 UTC (permalink / raw)
  To: Steve Poe <spoe@sfnet.cc>; +Cc: pgsql-performance

I'd use two of your drives to create a mirrored partition where pg_xlog 
resides separate from the actual data.

RAID 10 is probably appropriate for the remaining drives.

Fortunately, you're not using Dell, so you don't have to worry about 
the Perc3/Di RAID controller, which is not so compatible with Linux...

-tfo

  --
  Thomas F. O'Connell
  Co-Founder, Information Architect
  Sitening, LLC
  http://www.sitening.com/
  110 30th Avenue North, Suite 6
  Nashville, TN 37203-6320
  615-260-0005

On Mar 31, 2005, at 9:01 PM, Steve Poe wrote:

> Thanks for everyone's feedback on to best improve our Postgresql 
> database for the animal hospital. I re-read the PostgreSQL 8.0 
> Performance Checklist just to keep focused.
>
> We purchased (2) 4 x 146GB 10,000rpm SCSI U320 SCA drive arrays 
> ($2600) and (1) Sun W2100z dual AMD64 workstation with 4GB RAM 
> ($2500). We did not need a rack-mount server, so I though Sun's 
> workstation would do fine. I'll double the RAM. Hopefully, this should 
> out-perform our dual 2.8 Xeon with 4GB of RAM.
>
> Now, we need to purchase a good U320 RAID card now. Any suggestions 
> for those  which run well under Linux?
>
> These two drive arrays main purpose is for our database. For those 
> messed with drive arrays before, how would you slice-up the drive 
> array? Will database performance be effected how our RAID10 is 
> configured? Any suggestions?
>
> Thanks.
>
> Steve Poe




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
  2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
@ 2005-04-06 04:44   ` Kevin Brown <kevin@sysexperts.com>
  2005-04-06 11:28     ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
  2005-04-07 04:40     ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
  0 siblings, 2 replies; 9+ messages in thread

From: Kevin Brown @ 2005-04-06 04:44 UTC (permalink / raw)
  To: pgsql-performance

Thomas F.O'Connell wrote:
> I'd use two of your drives to create a mirrored partition where pg_xlog 
> resides separate from the actual data.
> 
> RAID 10 is probably appropriate for the remaining drives.
> 
> Fortunately, you're not using Dell, so you don't have to worry about 
> the Perc3/Di RAID controller, which is not so compatible with
> Linux...

Hmm...I have to wonder how true this is these days.

My company has a Dell 2500 with a Perc3/Di running Debian Linux, with
the 2.6.10 kernel.  The controller seems to work reasonably well,
though I wouldn't doubt that it's slower than a different one might
be.  But so far we haven't had any reliability issues with it.

Now, the performance is pretty bad considering the setup -- a RAID 5
with five 73.6 gig SCSI disks (10K RPM, I believe).  Reads through the
filesystem come through at about 65 megabytes/sec, writes about 35
megabytes/sec (at least, so says "bonnie -s 8192").  This is on a
system with a single 3 GHz Xeon and 1 gigabyte of memory.  I'd expect
much better read performance from what is essentially a stripe of 4
fast SCSI disks.


While compatibility hasn't really been an issue, at least as far as
the basics go, I still agree with your general sentiment -- stay away
from the Dells, at least if they have the Perc3/Di controller.  You'll
probably get much better performance out of something else.


-- 
Kevin Brown					      kevin@sysexperts.com



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
  2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
  2005-04-06 04:44   ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
@ 2005-04-06 11:28     ` Steinar H. Gunderson <sgunderson@bigfoot.com>
  2005-04-06 13:26       ` Re: Follow-Up: How to improve db performance with $7K? PFC <lists@boutiquenumerique.com>
  1 sibling, 1 reply; 9+ messages in thread

From: Steinar H. Gunderson @ 2005-04-06 11:28 UTC (permalink / raw)
  To: pgsql-performance

On Tue, Apr 05, 2005 at 09:44:56PM -0700, Kevin Brown wrote:
> Now, the performance is pretty bad considering the setup -- a RAID 5
> with five 73.6 gig SCSI disks (10K RPM, I believe).  Reads through the
> filesystem come through at about 65 megabytes/sec, writes about 35
> megabytes/sec (at least, so says "bonnie -s 8192").  This is on a
> system with a single 3 GHz Xeon and 1 gigabyte of memory.  I'd expect
> much better read performance from what is essentially a stripe of 4
> fast SCSI disks.

Data point here: We have a Linux software RAID quite close to the setup you
describe, with an onboard Adaptec controller and four 146GB 10000rpm disks,
and we get about 65MB/sec sustained when writing to an ext3 filesystem
(actually, when wgetting a file off the gigabit LAN :-) ). I haven't tested
reading, though.

/* Steinar */
-- 
Homepage: http://www.sesse.net/



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
  2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
  2005-04-06 04:44   ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
  2005-04-06 11:28     ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
@ 2005-04-06 13:26       ` PFC <lists@boutiquenumerique.com>
  2005-04-06 13:33         ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
  0 siblings, 1 reply; 9+ messages in thread

From: PFC @ 2005-04-06 13:26 UTC (permalink / raw)
  To: Steinar H. Gunderson <sgunderson@bigfoot.com>; pgsql-performance


> and we get about 65MB/sec sustained when writing to an ext3 filesystem
> (actually, when wgetting a file off the gigabit LAN :-) ). I haven't

	Well, unless you have PCI 64 bits, the "standard" PCI does 133 MB/s which  
is then split exactly in two times 66.5 MB/s for 1) reading from the PCI  
network card and 2) writing to the PCI harddisk controller. No wonder you  
get this figure, you're able to saturate your PCI bus, but it does not  
tell you a thing on the performance of your disk or network card... Note  
that the server which serves the file is limited in the same way unless  
the file is in cache (RAM) or it's PCI64. So...


> tested
> reading, though.
>
> /* Steinar */





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
  2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
  2005-04-06 04:44   ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
  2005-04-06 11:28     ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
  2005-04-06 13:26       ` Re: Follow-Up: How to improve db performance with $7K? PFC <lists@boutiquenumerique.com>
@ 2005-04-06 13:33         ` Steinar H. Gunderson <sgunderson@bigfoot.com>
  0 siblings, 0 replies; 9+ messages in thread

From: Steinar H. Gunderson @ 2005-04-06 13:33 UTC (permalink / raw)
  To: pgsql-performance

On Wed, Apr 06, 2005 at 03:26:33PM +0200, PFC wrote:
> 	Well, unless you have PCI 64 bits, the "standard" PCI does 133 MB/s 
> 	which  is then split exactly in two times 66.5 MB/s for 1) reading from the 
> PCI  network card and 2) writing to the PCI harddisk controller. No wonder 
> you  get this figure, you're able to saturate your PCI bus, but it does not 
> tell you a thing on the performance of your disk or network card... Note  
> that the server which serves the file is limited in the same way unless  
> the file is in cache (RAM) or it's PCI64. So...

This is PCI-X.

/* Steinar */
-- 
Homepage: http://www.sesse.net/



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
  2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
  2005-04-06 04:44   ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
@ 2005-04-07 04:40     ` Thomas F.O'Connell <tfo@sitening.com>
  1 sibling, 0 replies; 9+ messages in thread

From: Thomas F.O'Connell @ 2005-04-07 04:40 UTC (permalink / raw)
  To: Kevin Brown <kevin@sysexperts.com>; +Cc: pgsql-performance

Things might've changed somewhat over the past year, but this is from 
_the_ Linux guy at Dell...

-tfo

--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC

Strategic Open Source — Open Your i™

http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005


Date: Mon, 26 Apr 2004 14:15:02 -0500
From: Matt Domsch <Matt_Domsch@dell.com>
To: linux-poweredge@dell.com
Subject: PERC3/Di failure workaround hypothesis


--uXxzq0nDebZQVNAZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 26, 2004 at 11:10:36AM -0500, Sellek, Greg wrote:
 > Short of ordering a Perc4 for every 2650 that I want to upgrade to RH
 > ES, is there anything else I can do to get around the Perc3/Di
 > problem?

Our working hypothesis for a workaround is to do as follows:

In afacli, set:

Read Cache: enabled
Write Cache: enabled when protected

Then unplug the ROMB battery.  A reboot is not necessary.  The firmware 
will immediately drop into Write-Through Cache mode, which in our 
testing has not exhibited the problem.  Setting the write cache to 
disabled in afacli doesn't seem to help - you've got to unplug the 
battery with it in the above settings.

We are continuing to search for the root cause to the problem, and will 
update the list when we can.

Thanks,
Matt

--
Matt Domsch
Sr. Software Engineer, Lead Engineer
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

On Apr 5, 2005, at 11:44 PM, Kevin Brown wrote:

> Thomas F.O'Connell wrote:
>> I'd use two of your drives to create a mirrored partition where 
>> pg_xlog
>> resides separate from the actual data.
>>
>> RAID 10 is probably appropriate for the remaining drives.
>>
>> Fortunately, you're not using Dell, so you don't have to worry about
>> the Perc3/Di RAID controller, which is not so compatible with
>> Linux...
>
> Hmm...I have to wonder how true this is these days.
>
> My company has a Dell 2500 with a Perc3/Di running Debian Linux, with
> the 2.6.10 kernel.  The controller seems to work reasonably well,
> though I wouldn't doubt that it's slower than a different one might
> be.  But so far we haven't had any reliability issues with it.
>
> Now, the performance is pretty bad considering the setup -- a RAID 5
> with five 73.6 gig SCSI disks (10K RPM, I believe).  Reads through the
> filesystem come through at about 65 megabytes/sec, writes about 35
> megabytes/sec (at least, so says "bonnie -s 8192").  This is on a
> system with a single 3 GHz Xeon and 1 gigabyte of memory.  I'd expect
> much better read performance from what is essentially a stripe of 4
> fast SCSI disks.
>
>
> While compatibility hasn't really been an issue, at least as far as
> the basics go, I still agree with your general sentiment -- stay away
> from the Dells, at least if they have the Perc3/Di controller.  You'll
> probably get much better performance out of something else.
>
>
> -- 
> Kevin Brown					      kevin@sysexperts.com



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
@ 2005-04-01 21:23 ` Vivek Khera <vivek@khera.org>
  2005-04-02 16:21   ` Re: Follow-Up: How to improve db performance with $7K? Will LaShell <will@lashell.net>
  1 sibling, 1 reply; 9+ messages in thread

From: Vivek Khera @ 2005-04-01 21:23 UTC (permalink / raw)
  To: pgsql-performance


On Mar 31, 2005, at 9:01 PM, Steve Poe wrote:

> Now, we need to purchase a good U320 RAID card now. Any suggestions 
> for those  which run well under Linux?

Not sure if it works with linux, but under FreeBSD 5, the LSI MegaRAID 
cards are well supported.  You should be able to pick up a 320-2X with 
128Mb battery backed cache for about $1k.  Wicked fast... I'm suprized 
you didn't go for the 15k RPM drives for a small extra cost.




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Follow-Up: How to improve db performance with $7K?
  2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
  2005-04-01 21:23 ` Re: Follow-Up: How to improve db performance with $7K? Vivek Khera <vivek@khera.org>
@ 2005-04-02 16:21   ` Will LaShell <will@lashell.net>
  0 siblings, 0 replies; 9+ messages in thread

From: Will LaShell @ 2005-04-02 16:21 UTC (permalink / raw)
  To: Vivek Khera <vivek@khera.org>; +Cc: pgsql-performance

Vivek Khera wrote:

>
> On Mar 31, 2005, at 9:01 PM, Steve Poe wrote:
>
>> Now, we need to purchase a good U320 RAID card now. Any suggestions 
>> for those  which run well under Linux?
>
>
> Not sure if it works with linux, but under FreeBSD 5, the LSI MegaRAID 
> cards are well supported.  You should be able to pick up a 320-2X with 
> 128Mb battery backed cache for about $1k.  Wicked fast... I'm suprized 
> you didn't go for the 15k RPM drives for a small extra cost.



Wow, okay,  so  I'm not sure where everyone's email went,  but I got 
over a weeks worth of list emails at once. 

Several of you have sent me requests on where we purchased our systems 
at.  Compsource was the vendor,   www.c-source.com  or 
www.compsource.com.    The sales rep   we have is  Steve Taylor  or you 
can talk to the sales manager  Tom.    I've bought hardware from them 
for the last 2 years and I've been very pleased.  I'm sorry wasn't able 
to respond sooner.


Steve,  The LSI MegaRAID cards are where its at.  I've had -great- luck 
with them over the years. There were a few weird problems with a series 
awhile back where the linux driver needed tweaked by the developers  
along with a new bios update.  The 320 series is just as Vivek said, 
wicked fast. Very strong cards.  Be sure though when you order it to 
specificy the battery backup either with it,  or make sure you buy the 
right one for it. There are a couple of options with battery cache on 
the cards that can trip you up.

Good luck on your systems!   Now that I've got my email problems 
resolved  I'm definitely more than help to give any information you all 
need.



^ permalink  raw  reply  [nested|flat] 9+ messages in thread


end of thread, other threads:[~2005-04-07 04:40 UTC | newest]

Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44   ` Kevin Brown <kevin@sysexperts.com>
2005-04-06 11:28     ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-06 13:26       ` PFC <lists@boutiquenumerique.com>
2005-04-06 13:33         ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-07 04:40     ` Thomas F.O'Connell <tfo@sitening.com>
2005-04-01 21:23 ` Vivek Khera <vivek@khera.org>
2005-04-02 16:21   ` Will LaShell <will@lashell.net>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox