pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedFollow-Up: How to improve db performance with $7K?
9+ messages / 7 participants
[nested] [flat]
* Follow-Up: How to improve db performance with $7K?
@ 2005-04-01 02:01 Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
2005-04-01 21:23 ` Re: Follow-Up: How to improve db performance with $7K? Vivek Khera <vivek@khera.org>
0 siblings, 2 replies; 9+ messages in thread
From: Steve Poe @ 2005-04-01 02:01 UTC (permalink / raw)
To: ; +Cc: pgsql-performance
Thanks for everyone's feedback on to best improve our Postgresql
database for the animal hospital. I re-read the PostgreSQL 8.0
Performance Checklist just to keep focused.
We purchased (2) 4 x 146GB 10,000rpm SCSI U320 SCA drive arrays ($2600)
and (1) Sun W2100z dual AMD64 workstation with 4GB RAM ($2500). We did
not need a rack-mount server, so I though Sun's workstation would do
fine. I'll double the RAM. Hopefully, this should out-perform our dual
2.8 Xeon with 4GB of RAM.
Now, we need to purchase a good U320 RAID card now. Any suggestions for
those which run well under Linux?
These two drive arrays main purpose is for our database. For those
messed with drive arrays before, how would you slice-up the drive array?
Will database performance be effected how our RAID10 is configured? Any
suggestions?
Thanks.
Steve Poe
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
@ 2005-04-01 08:09 ` Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44 ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
1 sibling, 1 reply; 9+ messages in thread
From: Thomas F.O'Connell @ 2005-04-01 08:09 UTC (permalink / raw)
To: Steve Poe <spoe@sfnet.cc>; +Cc: pgsql-performance
I'd use two of your drives to create a mirrored partition where pg_xlog
resides separate from the actual data.
RAID 10 is probably appropriate for the remaining drives.
Fortunately, you're not using Dell, so you don't have to worry about
the Perc3/Di RAID controller, which is not so compatible with Linux...
-tfo
--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC
http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005
On Mar 31, 2005, at 9:01 PM, Steve Poe wrote:
> Thanks for everyone's feedback on to best improve our Postgresql
> database for the animal hospital. I re-read the PostgreSQL 8.0
> Performance Checklist just to keep focused.
>
> We purchased (2) 4 x 146GB 10,000rpm SCSI U320 SCA drive arrays
> ($2600) and (1) Sun W2100z dual AMD64 workstation with 4GB RAM
> ($2500). We did not need a rack-mount server, so I though Sun's
> workstation would do fine. I'll double the RAM. Hopefully, this should
> out-perform our dual 2.8 Xeon with 4GB of RAM.
>
> Now, we need to purchase a good U320 RAID card now. Any suggestions
> for those which run well under Linux?
>
> These two drive arrays main purpose is for our database. For those
> messed with drive arrays before, how would you slice-up the drive
> array? Will database performance be effected how our RAID10 is
> configured? Any suggestions?
>
> Thanks.
>
> Steve Poe
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
@ 2005-04-06 04:44 ` Kevin Brown <kevin@sysexperts.com>
2005-04-06 11:28 ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-07 04:40 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
0 siblings, 2 replies; 9+ messages in thread
From: Kevin Brown @ 2005-04-06 04:44 UTC (permalink / raw)
To: pgsql-performance
Thomas F.O'Connell wrote:
> I'd use two of your drives to create a mirrored partition where pg_xlog
> resides separate from the actual data.
>
> RAID 10 is probably appropriate for the remaining drives.
>
> Fortunately, you're not using Dell, so you don't have to worry about
> the Perc3/Di RAID controller, which is not so compatible with
> Linux...
Hmm...I have to wonder how true this is these days.
My company has a Dell 2500 with a Perc3/Di running Debian Linux, with
the 2.6.10 kernel. The controller seems to work reasonably well,
though I wouldn't doubt that it's slower than a different one might
be. But so far we haven't had any reliability issues with it.
Now, the performance is pretty bad considering the setup -- a RAID 5
with five 73.6 gig SCSI disks (10K RPM, I believe). Reads through the
filesystem come through at about 65 megabytes/sec, writes about 35
megabytes/sec (at least, so says "bonnie -s 8192"). This is on a
system with a single 3 GHz Xeon and 1 gigabyte of memory. I'd expect
much better read performance from what is essentially a stripe of 4
fast SCSI disks.
While compatibility hasn't really been an issue, at least as far as
the basics go, I still agree with your general sentiment -- stay away
from the Dells, at least if they have the Perc3/Di controller. You'll
probably get much better performance out of something else.
--
Kevin Brown kevin@sysexperts.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44 ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
@ 2005-04-06 11:28 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-06 13:26 ` Re: Follow-Up: How to improve db performance with $7K? PFC <lists@boutiquenumerique.com>
1 sibling, 1 reply; 9+ messages in thread
From: Steinar H. Gunderson @ 2005-04-06 11:28 UTC (permalink / raw)
To: pgsql-performance
On Tue, Apr 05, 2005 at 09:44:56PM -0700, Kevin Brown wrote:
> Now, the performance is pretty bad considering the setup -- a RAID 5
> with five 73.6 gig SCSI disks (10K RPM, I believe). Reads through the
> filesystem come through at about 65 megabytes/sec, writes about 35
> megabytes/sec (at least, so says "bonnie -s 8192"). This is on a
> system with a single 3 GHz Xeon and 1 gigabyte of memory. I'd expect
> much better read performance from what is essentially a stripe of 4
> fast SCSI disks.
Data point here: We have a Linux software RAID quite close to the setup you
describe, with an onboard Adaptec controller and four 146GB 10000rpm disks,
and we get about 65MB/sec sustained when writing to an ext3 filesystem
(actually, when wgetting a file off the gigabit LAN :-) ). I haven't tested
reading, though.
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44 ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
2005-04-06 11:28 ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
@ 2005-04-06 13:26 ` PFC <lists@boutiquenumerique.com>
2005-04-06 13:33 ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
0 siblings, 1 reply; 9+ messages in thread
From: PFC @ 2005-04-06 13:26 UTC (permalink / raw)
To: Steinar H. Gunderson <sgunderson@bigfoot.com>; pgsql-performance
> and we get about 65MB/sec sustained when writing to an ext3 filesystem
> (actually, when wgetting a file off the gigabit LAN :-) ). I haven't
Well, unless you have PCI 64 bits, the "standard" PCI does 133 MB/s which
is then split exactly in two times 66.5 MB/s for 1) reading from the PCI
network card and 2) writing to the PCI harddisk controller. No wonder you
get this figure, you're able to saturate your PCI bus, but it does not
tell you a thing on the performance of your disk or network card... Note
that the server which serves the file is limited in the same way unless
the file is in cache (RAM) or it's PCI64. So...
> tested
> reading, though.
>
> /* Steinar */
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44 ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
2005-04-06 11:28 ` Re: Follow-Up: How to improve db performance with $7K? Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-06 13:26 ` Re: Follow-Up: How to improve db performance with $7K? PFC <lists@boutiquenumerique.com>
@ 2005-04-06 13:33 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
0 siblings, 0 replies; 9+ messages in thread
From: Steinar H. Gunderson @ 2005-04-06 13:33 UTC (permalink / raw)
To: pgsql-performance
On Wed, Apr 06, 2005 at 03:26:33PM +0200, PFC wrote:
> Well, unless you have PCI 64 bits, the "standard" PCI does 133 MB/s
> which is then split exactly in two times 66.5 MB/s for 1) reading from the
> PCI network card and 2) writing to the PCI harddisk controller. No wonder
> you get this figure, you're able to saturate your PCI bus, but it does not
> tell you a thing on the performance of your disk or network card... Note
> that the server which serves the file is limited in the same way unless
> the file is in cache (RAM) or it's PCI64. So...
This is PCI-X.
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Re: Follow-Up: How to improve db performance with $7K? Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44 ` Re: Follow-Up: How to improve db performance with $7K? Kevin Brown <kevin@sysexperts.com>
@ 2005-04-07 04:40 ` Thomas F.O'Connell <tfo@sitening.com>
1 sibling, 0 replies; 9+ messages in thread
From: Thomas F.O'Connell @ 2005-04-07 04:40 UTC (permalink / raw)
To: Kevin Brown <kevin@sysexperts.com>; +Cc: pgsql-performance
Things might've changed somewhat over the past year, but this is from
_the_ Linux guy at Dell...
-tfo
--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC
Strategic Open Source — Open Your i™
http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005
Date: Mon, 26 Apr 2004 14:15:02 -0500
From: Matt Domsch <Matt_Domsch@dell.com>
To: linux-poweredge@dell.com
Subject: PERC3/Di failure workaround hypothesis
--uXxzq0nDebZQVNAZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Apr 26, 2004 at 11:10:36AM -0500, Sellek, Greg wrote:
> Short of ordering a Perc4 for every 2650 that I want to upgrade to RH
> ES, is there anything else I can do to get around the Perc3/Di
> problem?
Our working hypothesis for a workaround is to do as follows:
In afacli, set:
Read Cache: enabled
Write Cache: enabled when protected
Then unplug the ROMB battery. A reboot is not necessary. The firmware
will immediately drop into Write-Through Cache mode, which in our
testing has not exhibited the problem. Setting the write cache to
disabled in afacli doesn't seem to help - you've got to unplug the
battery with it in the above settings.
We are continuing to search for the root cause to the problem, and will
update the list when we can.
Thanks,
Matt
--
Matt Domsch
Sr. Software Engineer, Lead Engineer
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com
On Apr 5, 2005, at 11:44 PM, Kevin Brown wrote:
> Thomas F.O'Connell wrote:
>> I'd use two of your drives to create a mirrored partition where
>> pg_xlog
>> resides separate from the actual data.
>>
>> RAID 10 is probably appropriate for the remaining drives.
>>
>> Fortunately, you're not using Dell, so you don't have to worry about
>> the Perc3/Di RAID controller, which is not so compatible with
>> Linux...
>
> Hmm...I have to wonder how true this is these days.
>
> My company has a Dell 2500 with a Perc3/Di running Debian Linux, with
> the 2.6.10 kernel. The controller seems to work reasonably well,
> though I wouldn't doubt that it's slower than a different one might
> be. But so far we haven't had any reliability issues with it.
>
> Now, the performance is pretty bad considering the setup -- a RAID 5
> with five 73.6 gig SCSI disks (10K RPM, I believe). Reads through the
> filesystem come through at about 65 megabytes/sec, writes about 35
> megabytes/sec (at least, so says "bonnie -s 8192"). This is on a
> system with a single 3 GHz Xeon and 1 gigabyte of memory. I'd expect
> much better read performance from what is essentially a stripe of 4
> fast SCSI disks.
>
>
> While compatibility hasn't really been an issue, at least as far as
> the basics go, I still agree with your general sentiment -- stay away
> from the Dells, at least if they have the Perc3/Di controller. You'll
> probably get much better performance out of something else.
>
>
> --
> Kevin Brown kevin@sysexperts.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
@ 2005-04-01 21:23 ` Vivek Khera <vivek@khera.org>
2005-04-02 16:21 ` Re: Follow-Up: How to improve db performance with $7K? Will LaShell <will@lashell.net>
1 sibling, 1 reply; 9+ messages in thread
From: Vivek Khera @ 2005-04-01 21:23 UTC (permalink / raw)
To: pgsql-performance
On Mar 31, 2005, at 9:01 PM, Steve Poe wrote:
> Now, we need to purchase a good U320 RAID card now. Any suggestions
> for those which run well under Linux?
Not sure if it works with linux, but under FreeBSD 5, the LSI MegaRAID
cards are well supported. You should be able to pick up a 320-2X with
128Mb battery backed cache for about $1k. Wicked fast... I'm suprized
you didn't go for the 15k RPM drives for a small extra cost.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Follow-Up: How to improve db performance with $7K?
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 21:23 ` Re: Follow-Up: How to improve db performance with $7K? Vivek Khera <vivek@khera.org>
@ 2005-04-02 16:21 ` Will LaShell <will@lashell.net>
0 siblings, 0 replies; 9+ messages in thread
From: Will LaShell @ 2005-04-02 16:21 UTC (permalink / raw)
To: Vivek Khera <vivek@khera.org>; +Cc: pgsql-performance
Vivek Khera wrote:
>
> On Mar 31, 2005, at 9:01 PM, Steve Poe wrote:
>
>> Now, we need to purchase a good U320 RAID card now. Any suggestions
>> for those which run well under Linux?
>
>
> Not sure if it works with linux, but under FreeBSD 5, the LSI MegaRAID
> cards are well supported. You should be able to pick up a 320-2X with
> 128Mb battery backed cache for about $1k. Wicked fast... I'm suprized
> you didn't go for the 15k RPM drives for a small extra cost.
Wow, okay, so I'm not sure where everyone's email went, but I got
over a weeks worth of list emails at once.
Several of you have sent me requests on where we purchased our systems
at. Compsource was the vendor, www.c-source.com or
www.compsource.com. The sales rep we have is Steve Taylor or you
can talk to the sales manager Tom. I've bought hardware from them
for the last 2 years and I've been very pleased. I'm sorry wasn't able
to respond sooner.
Steve, The LSI MegaRAID cards are where its at. I've had -great- luck
with them over the years. There were a few weird problems with a series
awhile back where the linux driver needed tweaked by the developers
along with a new bios update. The 320 series is just as Vivek said,
wicked fast. Very strong cards. Be sure though when you order it to
specificy the battery backup either with it, or make sure you buy the
right one for it. There are a couple of options with battery cache on
the cards that can trip you up.
Good luck on your systems! Now that I've got my email problems
resolved I'm definitely more than help to give any information you all
need.
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2005-04-07 04:40 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-04-01 02:01 Follow-Up: How to improve db performance with $7K? Steve Poe <spoe@sfnet.cc>
2005-04-01 08:09 ` Thomas F.O'Connell <tfo@sitening.com>
2005-04-06 04:44 ` Kevin Brown <kevin@sysexperts.com>
2005-04-06 11:28 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-06 13:26 ` PFC <lists@boutiquenumerique.com>
2005-04-06 13:33 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2005-04-07 04:40 ` Thomas F.O'Connell <tfo@sitening.com>
2005-04-01 21:23 ` Vivek Khera <vivek@khera.org>
2005-04-02 16:21 ` Will LaShell <will@lashell.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox