pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedData write speed
7+ messages / 5 participants
[nested] [flat]
* Data write speed
@ 2003-02-18 21:45 Josh Berkus <josh@agliodbs.com>
2003-02-18 22:22 ` Re: Data write speed Hannu Krosing <hannu@tm.ee>
2003-02-18 22:27 ` Re: Data write speed Kevin Brown <kevin@sysexperts.com>
0 siblings, 2 replies; 7+ messages in thread
From: Josh Berkus @ 2003-02-18 21:45 UTC (permalink / raw)
To: pgsql-performance
Folks,
I have a new system with an Adaptec 2200S RAID controller. I've been
testing some massive data transformations on it, and data write speed
seems to top out at 3mb/second .... which seems slow to me for this
kind of hardware. As it is, I have RAM and CPU sitting idle while they
wait for the disks to finish.
Thoughts, anyone?
-Josh
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Data write speed
2003-02-18 21:45 Data write speed Josh Berkus <josh@agliodbs.com>
@ 2003-02-18 22:22 ` Hannu Krosing <hannu@tm.ee>
1 sibling, 0 replies; 7+ messages in thread
From: Hannu Krosing @ 2003-02-18 22:22 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-performance
Josh Berkus kirjutas T, 18.02.2003 kell 23:45:
> Folks,
>
> I have a new system with an Adaptec 2200S RAID controller.
RAID what (0,1,1+0,5,...) ?
> I've been
> testing some massive data transformations on it, and data write speed
> seems to top out at 3mb/second .... which seems slow to me for this
> kind of hardware. As it is, I have RAM and CPU sitting idle while they
> wait for the disks to finish.
>
> Thoughts, anyone?
How have you tested it ?
What OS ?
---------------
Hannu
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Data write speed
2003-02-18 21:45 Data write speed Josh Berkus <josh@agliodbs.com>
@ 2003-02-18 22:27 ` Kevin Brown <kevin@sysexperts.com>
2003-02-18 23:51 ` Re: Data write speed Josh Berkus <josh@agliodbs.com>
1 sibling, 1 reply; 7+ messages in thread
From: Kevin Brown @ 2003-02-18 22:27 UTC (permalink / raw)
To: pgsql-performance
Josh Berkus wrote:
> Folks,
>
> I have a new system with an Adaptec 2200S RAID controller. I've been
> testing some massive data transformations on it, and data write speed
> seems to top out at 3mb/second .... which seems slow to me for this
> kind of hardware. As it is, I have RAM and CPU sitting idle while they
> wait for the disks to finish.
>
> Thoughts, anyone?
That does seem low. What rate do you get with software RAID (of the
same type, of course) to the same disks (might have to be through a
standard SCSI controller to be meaningful) with roughly the same
disks/channel distribution?
My experience with hardware RAID (at least with the hardware available
a few years ago) versus software RAID is that software RAID is almost
always going to be faster because RAID speed seems to be very
dependent on the speed of the RAID controller's CPU. And computer
systems usually have a processor that's significantly faster than the
processor on a hardware RAID controller. It's rare that an
application will be as CPU intensive as it is I/O intensive (in
particular, there are relatively few applications that will be burning
CPU at the same time they're waiting for I/O to complete), so the
faster you can get your I/O completed, the higher the overall
throughput will be even if you have to use some CPU to do the I/O.
That may have changed some since CPUs now are much faster than they
used to be, even on hardware RAID controllers, but to me that just
means that you can build a larger RAID system before saturating the
CPU.
The Adaptec 2200S has a 100MHz CPU. That's pretty weak. The typical
modern desktop system has a factor of 20 more CPU power than that. A
software RAID setup would have no trouble blowing the 2200S out of the
water, especially if the OS is able to make use of features such as
tagged queueing.
Since the 2200S has a JBOD mode, you might consider testing a software
RAID setup across that, just to see how much of a difference doing the
RAID calculations on the host system makes.
--
Kevin Brown kevin@sysexperts.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Data write speed
2003-02-18 21:45 Data write speed Josh Berkus <josh@agliodbs.com>
2003-02-18 22:27 ` Re: Data write speed Kevin Brown <kevin@sysexperts.com>
@ 2003-02-18 23:51 ` Josh Berkus <josh@agliodbs.com>
2003-02-19 01:21 ` Re: Data write speed Justin Clift <justin@postgresql.org>
0 siblings, 1 reply; 7+ messages in thread
From: Josh Berkus @ 2003-02-18 23:51 UTC (permalink / raw)
To: Kevin Brown <kevin@sysexperts.com>; pgsql-performance
Kevin,
> That does seem low. What rate do you get with software RAID (of the
> same type, of course) to the same disks (might have to be through a
> standard SCSI controller to be meaningful) with roughly the same
> disks/channel distribution?
Unfortunately, I can't do this without blowing away all of my current
software setup. I noticed the performance problem *after* installing
Linux, PostgreSQL, Apache, PHP, pam_auth, NIS, and my database ....
and, of course, customized builds of the above.
> The Adaptec 2200S has a 100MHz CPU. That's pretty weak. The typical
> modern desktop system has a factor of 20 more CPU power than that. A
> software RAID setup would have no trouble blowing the 2200S out of
> the
> water, especially if the OS is able to make use of features such as
> tagged queueing.
>
> Since the 2200S has a JBOD mode, you might consider testing a
> software
> RAID setup across that, just to see how much of a difference doing
> the
> RAID calculations on the host system makes.
This isn't something I can do without blowing away my current software
setup, hey?
-Josh
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Data write speed
2003-02-18 21:45 Data write speed Josh Berkus <josh@agliodbs.com>
2003-02-18 22:27 ` Re: Data write speed Kevin Brown <kevin@sysexperts.com>
2003-02-18 23:51 ` Re: Data write speed Josh Berkus <josh@agliodbs.com>
@ 2003-02-19 01:21 ` Justin Clift <justin@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Justin Clift @ 2003-02-19 01:21 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; +Cc: Kevin Brown <kevin@sysexperts.com>; pgsql-performance
Josh Berkus wrote:
<snip>
> This isn't something I can do without blowing away my current software
> setup, hey?
Hi Josh,
Any chance that it's a hardware problem causing a SCSI level slowdown?
i.e. termination accidentally turned on for a device that's not at the
end of a SCSI chain, or perhaps using a cable that's not of the right
spec? Stuff that will still let it work, but at a reduced rate. Maybe
even a setting in the Adaptec controller's BIOS? Might be worth looking
at dmesg and seeing what it reports the SCSI interface speed to be.
:-)
Regards and best wishes,
Justin Clift
> -Josh
--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Data write speed
@ 2003-02-27 16:57 Josh Berkus <josh@agliodbs.com>
2003-02-27 17:54 ` Re: Data write speed scott.marlowe <scott.marlowe@ihs.com>
0 siblings, 1 reply; 7+ messages in thread
From: Josh Berkus @ 2003-02-27 16:57 UTC (permalink / raw)
To: Murthy Kambhampaty <murthy.kambhampaty@goeci.com>; +Cc: pgsql-performance
Murthy,
> You could get mondo (http://www.microwerks.net/~hugo/), then backup your
> system to CDs and restore it with the new filesystem layout. You might want
> to do these backups as a matter of course?
Thanks for the suggestion. The problem isn't backup media ... we have a DLT
drive ... the problem is time. This particular application is already about
4 weeks behind schedule because of various hardware problems. At some point,
Kevin Brown and I will take a weekend to swap the postgres files to a spare
disk, and re-format the data array as pass-through Linux RAID.
And this is the last time I leave it up to the company sysadmin to buy
hardware for a database server, even with explicit instructions ... "Yes, I
saw which one you wanted, but the 2200S was on sale!"
--
Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Data write speed
2003-02-27 16:57 Re: Data write speed Josh Berkus <josh@agliodbs.com>
@ 2003-02-27 17:54 ` scott.marlowe <scott.marlowe@ihs.com>
0 siblings, 0 replies; 7+ messages in thread
From: scott.marlowe @ 2003-02-27 17:54 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; +Cc: Murthy Kambhampaty <murthy.kambhampaty@goeci.com>; pgsql-performance
On Thu, 27 Feb 2003, Josh Berkus wrote:
> Murthy,
>
> > You could get mondo (http://www.microwerks.net/~hugo/), then backup your
> > system to CDs and restore it with the new filesystem layout. You might want
> > to do these backups as a matter of course?
>
> Thanks for the suggestion. The problem isn't backup media ... we have a DLT
> drive ... the problem is time. This particular application is already about
> 4 weeks behind schedule because of various hardware problems. At some point,
> Kevin Brown and I will take a weekend to swap the postgres files to a spare
> disk, and re-format the data array as pass-through Linux RAID.
>
> And this is the last time I leave it up to the company sysadmin to buy
> hardware for a database server, even with explicit instructions ... "Yes, I
> saw which one you wanted, but the 2200S was on sale!"
I still remember going round and round with a hardware engineer who was
extolling the adaptec AIC 133 controller as a great raid controller. I
finally made him test it instead of just reading the pamphlet that came
with it... Needless to say, it couldn't hold it's own against a straight
symbios UW card running linux software RAID.
He's the same guy who speced my workstation with no AGP slot in it. The
week before he was laid off. Talk about bad timing... :-(
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2003-02-27 17:54 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-02-18 21:45 Data write speed Josh Berkus <josh@agliodbs.com>
2003-02-18 22:22 ` Hannu Krosing <hannu@tm.ee>
2003-02-18 22:27 ` Kevin Brown <kevin@sysexperts.com>
2003-02-18 23:51 ` Josh Berkus <josh@agliodbs.com>
2003-02-19 01:21 ` Justin Clift <justin@postgresql.org>
2003-02-27 16:57 Re: Data write speed Josh Berkus <josh@agliodbs.com>
2003-02-27 17:54 ` scott.marlowe <scott.marlowe@ihs.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox