pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedEasy read-heavy benchmark kicking around?
11+ messages / 9 participants
[nested] [flat]
* Easy read-heavy benchmark kicking around?
@ 2006-11-06 20:47 Brian Hurt <bhurt@janestcapital.com>
0 siblings, 4 replies; 11+ messages in thread
From: Brian Hurt @ 2006-11-06 20:47 UTC (permalink / raw)
To: pgsql-performance
I'm having a spot of problem with out storage device vendor. Read
performance (as measured by both bonnie++ and hdparm -t) is abysmal
(~14Mbyte/sec), and we're trying to get them to fix it. Unfortunately,
they're using the fact that bonnie++ is an open source benchmark to
weasle out of doing anything- they can't fix it unless I can show an
impact in Postgresql.
So the question is: is there an easy to install and run, read-heavy
benchmark out there that I can wave at them to get them to fix the
problem? I have a second database running on a single SATA drive, so I
can use that as a comparison point- "look, we're getting 1/3rd the read
speed of a single SATA drive- this sucks!"
Any advice?
Brian
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-06 20:53 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 0 replies; 11+ messages in thread
From: Luke Lonergan @ 2006-11-06 20:53 UTC (permalink / raw)
To: Brian Hurt <bhurt@janestcapital.com>; pgsql-performance
Select count(*) from table-twice-size-of-ram
Divide the query time by the number of pages in the table times the pagesize (normally 8KB) and you have your net disk rate.
- Luke
Msg is shrt cuz m on ma treo
-----Original Message-----
From: Brian Hurt [mailto:bhurt@janestcapital.com]
Sent: Monday, November 06, 2006 03:49 PM Eastern Standard Time
To: pgsql-performance@postgresql.org
Subject: [PERFORM] Easy read-heavy benchmark kicking around?
I'm having a spot of problem with out storage device vendor. Read
performance (as measured by both bonnie++ and hdparm -t) is abysmal
(~14Mbyte/sec), and we're trying to get them to fix it. Unfortunately,
they're using the fact that bonnie++ is an open source benchmark to
weasle out of doing anything- they can't fix it unless I can show an
impact in Postgresql.
So the question is: is there an easy to install and run, read-heavy
benchmark out there that I can wave at them to get them to fix the
problem? I have a second database running on a single SATA drive, so I
can use that as a comparison point- "look, we're getting 1/3rd the read
speed of a single SATA drive- this sucks!"
Any advice?
Brian
---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-06 21:09 Merlin Moncure <mmoncure@gmail.com>
parent: Brian Hurt <bhurt@janestcapital.com>
3 siblings, 1 reply; 11+ messages in thread
From: Merlin Moncure @ 2006-11-06 21:09 UTC (permalink / raw)
To: Brian Hurt <bhurt@janestcapital.com>; +Cc: pgsql-performance
On 11/6/06, Brian Hurt <bhurt@janestcapital.com> wrote:
> I'm having a spot of problem with out storage device vendor. Read
> performance (as measured by both bonnie++ and hdparm -t) is abysmal
> (~14Mbyte/sec), and we're trying to get them to fix it. Unfortunately,
> they're using the fact that bonnie++ is an open source benchmark to
> weasle out of doing anything- they can't fix it unless I can show an
> impact in Postgresql.
>
> So the question is: is there an easy to install and run, read-heavy
> benchmark out there that I can wave at them to get them to fix the
> problem? I have a second database running on a single SATA drive, so I
> can use that as a comparison point- "look, we're getting 1/3rd the read
> speed of a single SATA drive- this sucks!"
hitachi?
my experience with storage vendors is when they say things like that
they know full well their device completely sucks and are just
stalling so that you give up.
merlin
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-06 21:26 Scott Marlowe <smarlowe@g2switchworks.com>
parent: Merlin Moncure <mmoncure@gmail.com>
0 siblings, 0 replies; 11+ messages in thread
From: Scott Marlowe @ 2006-11-06 21:26 UTC (permalink / raw)
To: Merlin Moncure <mmoncure@gmail.com>; +Cc: Brian Hurt <bhurt@janestcapital.com>; pgsql-performance
On Mon, 2006-11-06 at 15:09, Merlin Moncure wrote:
> On 11/6/06, Brian Hurt <bhurt@janestcapital.com> wrote:
> > I'm having a spot of problem with out storage device vendor. Read
> > performance (as measured by both bonnie++ and hdparm -t) is abysmal
> > (~14Mbyte/sec), and we're trying to get them to fix it. Unfortunately,
> > they're using the fact that bonnie++ is an open source benchmark to
> > weasle out of doing anything- they can't fix it unless I can show an
> > impact in Postgresql.
> >
> > So the question is: is there an easy to install and run, read-heavy
> > benchmark out there that I can wave at them to get them to fix the
> > problem? I have a second database running on a single SATA drive, so I
> > can use that as a comparison point- "look, we're getting 1/3rd the read
> > speed of a single SATA drive- this sucks!"
>
> hitachi?
>
> my experience with storage vendors is when they say things like that
> they know full well their device completely sucks and are just
> stalling so that you give up.
Man, if I were the OP I'd be naming names, and letting the idiots at
INSERT MAJOR VENDOR HERE know that I was naming names to the whole of
the postgresql community and open source as well to make the point that
if they look down on open source so much, then open source should look
down on them.
PostgreSQL is open source software, BSD and Linux are open source / free
software. bonnie++'s licensing shouldn't matter one nit, and I'd let
everyone know how shittily I was being treated by this vendor until
their fixed their crap or took it back.
Note that if you're using fibre channel etc... the problem might well be
in your own hardware / device drivers. There are a lot of real crap FC
and relative cards out there.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-07 01:52 Mark Kirkwood <markir@paradise.net.nz>
parent: Brian Hurt <bhurt@janestcapital.com>
3 siblings, 0 replies; 11+ messages in thread
From: Mark Kirkwood @ 2006-11-07 01:52 UTC (permalink / raw)
To: Brian Hurt <bhurt@janestcapital.com>; +Cc: pgsql-performance
Brian Hurt wrote:
> I'm having a spot of problem with out storage device vendor. Read
> performance (as measured by both bonnie++ and hdparm -t) is abysmal
> (~14Mbyte/sec), and we're trying to get them to fix it. Unfortunately,
> they're using the fact that bonnie++ is an open source benchmark to
> weasle out of doing anything- they can't fix it unless I can show an
> impact in Postgresql.
>
> So the question is: is there an easy to install and run, read-heavy
> benchmark out there that I can wave at them to get them to fix the
> problem? I have a second database running on a single SATA drive, so I
> can use that as a comparison point- "look, we're getting 1/3rd the read
> speed of a single SATA drive- this sucks!"
>
You could use the lineitem table from the TPC-H dataset
(http://www.tpc.org/tpch/default.asp).
Generate the dataset for a scale factor that makes lineitem about 2x
your ram, load the table and do:
SELECT count(*) FROM lineitem
vmstat or iostat while this is happening should display your meager
throughput well enough to get your vendors attention (I'm checking this
on a fairly old 4 disk system of mine as I type this - I'm seeing about
90Mb/s...)
best wishes
Mark
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-07 10:49 Dimitri Fontaine <dim@dalibo.com>
parent: Brian Hurt <bhurt@janestcapital.com>
3 siblings, 0 replies; 11+ messages in thread
From: Dimitri Fontaine @ 2006-11-07 10:49 UTC (permalink / raw)
To: pgsql-performance
Hi,
Le lundi 6 novembre 2006 21:47, Brian Hurt a écrit :
> So the question is: is there an easy to install and run, read-heavy
> benchmark out there that I can wave at them to get them to fix the
> problem? I have a second database running on a single SATA drive, so I
> can use that as a comparison point- "look, we're getting 1/3rd the read
> speed of a single SATA drive- this sucks!"
>
> Any advice?
Tsung is an easy to use open-source multi-protocol distributed load testing
tool. The more simple way to use it to stress test your machine would be
using pgfouine to setup a test from postgresql logs:
http://tsung.erlang-projects.org/
http://pgfouine.projects.postgresql.org/tsung.html
Tsung will easily simulate a lot (more than 1000) of concurrent users playing
your custom defined load scenario. Then it will give you some plots to
analyze results, for disk io though you'll have to use some other tool(s)
while tests are running.
Regards,
--
Dimitri Fontaine
http://www.dalibo.com/
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-08 16:18 Markus Schaber <schabi@logix-tt.com>
parent: Brian Hurt <bhurt@janestcapital.com>
3 siblings, 1 reply; 11+ messages in thread
From: Markus Schaber @ 2006-11-08 16:18 UTC (permalink / raw)
To: Brian Hurt <bhurt@janestcapital.com>; +Cc: pgsql-performance
Hi, Brian,
Brian Hurt wrote:
> So the question is: is there an easy to install and run, read-heavy
> benchmark out there that I can wave at them to get them to fix the
> problem?
For sequential read performance, use dd. Most variants of dd I've seen
output some timing information, and if not, do a "time dd
if=/your/device of=/dev/null bs=1M" on the partition.
Markus
--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf. | Software Development GIS
Fight against software patents in Europe! www.ffii.org
www.nosoftwarepatents.org
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-08 16:34 Merlin Moncure <mmoncure@gmail.com>
parent: Markus Schaber <schabi@logix-tt.com>
0 siblings, 2 replies; 11+ messages in thread
From: Merlin Moncure @ 2006-11-08 16:34 UTC (permalink / raw)
To: Markus Schaber <schabi@logix-tt.com>; +Cc: Brian Hurt <bhurt@janestcapital.com>; pgsql-performance
On 11/8/06, Markus Schaber <schabi@logix-tt.com> wrote:
> Hi, Brian,
>
> Brian Hurt wrote:
>
> > So the question is: is there an easy to install and run, read-heavy
> > benchmark out there that I can wave at them to get them to fix the
> > problem?
>
> For sequential read performance, use dd. Most variants of dd I've seen
> output some timing information, and if not, do a "time dd
> if=/your/device of=/dev/null bs=1M" on the partition.
we had a similar problem with a hitachi san, the ams200. Their
performance group refused to admit the fact that 50mb/sec dd test was
a valid performance benchmark and needed to be addressed. Yes, that
was a HITACHI SAN, the AMS200, which hitachi's performance group
claimed was 'acceptable performance'. This was the advice we got
after swapping out all the hardware and buying an entitlement to
redhat enterprise which we had to do to get them to talk to us.
oh, the unit also lost a controller after about a week of
operation...the unit being a HITACHI SAN, the AMS200.
any questions?
merlin
p.s. we have had good experiences with the adtx.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-08 16:57 Cosimo Streppone <cosimo@streppone.it>
parent: Merlin Moncure <mmoncure@gmail.com>
1 sibling, 0 replies; 11+ messages in thread
From: Cosimo Streppone @ 2006-11-08 16:57 UTC (permalink / raw)
To: pgsql-performance
Merlin Moncure wrote:
> On 11/8/06, Markus Schaber <schabi@logix-tt.com> wrote:
>> Hi, Brian,
>>
>> Brian Hurt wrote:
>>
>> > So the question is: is there an easy to install and run, read-heavy
>> > benchmark out there that I can wave at them to get them to fix the
>> > problem?
>>
>> For sequential read performance, use dd. Most variants of dd I've seen
>> output some timing information, and if not, do a "time dd
>> if=/your/device of=/dev/null bs=1M" on the partition.
>
> we had a similar problem with a hitachi san, the ams200. Their
> performance group refused to admit the fact that 50mb/sec dd test was
> a valid performance benchmark and needed to be addressed.
>
> [...]
>
> oh, the unit also lost a controller after about a week of
> operation...the unit being a HITACHI SAN, the AMS200.
>
> any questions?
Yes, one.
What was that unit?
;-)
--
Cosimo
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-08 19:32 Luke Lonergan <llonergan@greenplum.com>
parent: Merlin Moncure <mmoncure@gmail.com>
1 sibling, 0 replies; 11+ messages in thread
From: Luke Lonergan @ 2006-11-08 19:32 UTC (permalink / raw)
To: Merlin Moncure <mmoncure@gmail.com>; Markus Schaber <schabi@logix-tt.com>; +Cc: Brian Hurt <bhurt@janestcapital.com>; pgsql-performance
Similar experiences with HP and their SmartArray 5i controller on Linux.
The answer was: "this controller has won awards for performance! It can't be
slow!", so we made them test it in their own labs an prove just how awfully
slow it was. In the case of the 5i, it became apparent that HP had no
internal expertise on Linux and their controllers, the driver was built by a
third party that they didn't support and their performance people didn't
deal with the 5i at all.
In the end, all manner of benchmarks after you've purchased aren't a good
substitute for the up front question: do you have documentation of the
performance of your RAID controller on [Linux, Solaris, ...]?
I would like everyone who purchases IBM, Dell, HP or Sun to demand that
documentation - then perhaps we'd see higher quality drivers and hardware
result.
- Luke
On 11/8/06 8:34 AM, "Merlin Moncure" <mmoncure@gmail.com> wrote:
> On 11/8/06, Markus Schaber <schabi@logix-tt.com> wrote:
>> Hi, Brian,
>>
>> Brian Hurt wrote:
>>
>>> So the question is: is there an easy to install and run, read-heavy
>>> benchmark out there that I can wave at them to get them to fix the
>>> problem?
>>
>> For sequential read performance, use dd. Most variants of dd I've seen
>> output some timing information, and if not, do a "time dd
>> if=/your/device of=/dev/null bs=1M" on the partition.
>
> we had a similar problem with a hitachi san, the ams200. Their
> performance group refused to admit the fact that 50mb/sec dd test was
> a valid performance benchmark and needed to be addressed. Yes, that
> was a HITACHI SAN, the AMS200, which hitachi's performance group
> claimed was 'acceptable performance'. This was the advice we got
> after swapping out all the hardware and buying an entitlement to
> redhat enterprise which we had to do to get them to talk to us.
>
> oh, the unit also lost a controller after about a week of
> operation...the unit being a HITACHI SAN, the AMS200.
>
> any questions?
>
> merlin
>
> p.s. we have had good experiences with the adtx.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Easy read-heavy benchmark kicking around?
@ 2006-11-09 17:28 Merlin Moncure <mmoncure@gmail.com>
0 siblings, 0 replies; 11+ messages in thread
From: Merlin Moncure @ 2006-11-09 17:28 UTC (permalink / raw)
To: Spiegelberg, Greg <gspiegelberg@cranel.com>; +Cc: pgsql-performance
On 11/8/06, Spiegelberg, Greg <gspiegelberg@cranel.com> wrote:
> Merlin,
>
> I'm kinda shocked you had such a bad exp. with the AMS200. We have a
> unit here hooked up to a 4-node Linux cluster with 4 databases banging
> on it and we get good, consistent perfomance out of it. All 4 nodes can
> throw 25 to 75 MB/s simultaneously without a hiccup.
>
> I'm curious, what was your AMS, server and SAN config?
we had quad opteron 870 in a sun v40z. two trays of 400g sata drives
and the 4 15k fc drives they make you buy. o/s was originally gentoo
and emulex but we switched to redhat as4/qlogic to get support from
them.
the highest performance we ever got was around 120mb/sec writing to
the 4 fc drives in raid 10. however, the sata's could not even do 100
and for some reason when we added a second raid group the performance
dropped 40% for a reason that their performance group could not
explain. compounding the problem was that our assigned tech did not
know linux and there was a one week turnaround to get support emails
answered. Their sales and support staff were snotty and completely
unhelpful. Also we had to do a complex migration process which
involved physically moving the unit to multiple racks for data
transfer which we were going to have to coordinate with hitachi
support because they do not allow you to rack/unrack your own unit.
ultimately, we returned the unit and bought a adtx san. for less than
half the price of the hitachi, we got a dual 4gb controller mixed
sata/sas that supports 750g sata drives. It also has sas ports in the
back for direct attachment to sas hba. in an active/active
configuration, the unit can sustain 500mb/sec, and has 50% more
storage in 1/3 the rack space.
melrin
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2006-11-09 17:28 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-11-06 20:47 Easy read-heavy benchmark kicking around? Brian Hurt <bhurt@janestcapital.com>
2006-11-06 21:09 ` Merlin Moncure <mmoncure@gmail.com>
2006-11-06 21:26 ` Scott Marlowe <smarlowe@g2switchworks.com>
2006-11-07 01:52 ` Mark Kirkwood <markir@paradise.net.nz>
2006-11-07 10:49 ` Dimitri Fontaine <dim@dalibo.com>
2006-11-08 16:18 ` Markus Schaber <schabi@logix-tt.com>
2006-11-08 16:34 ` Merlin Moncure <mmoncure@gmail.com>
2006-11-08 16:57 ` Cosimo Streppone <cosimo@streppone.it>
2006-11-08 19:32 ` Luke Lonergan <llonergan@greenplum.com>
2006-11-06 20:53 Re: Easy read-heavy benchmark kicking around? Luke Lonergan <LLonergan@greenplum.com>
2006-11-09 17:28 Re: Easy read-heavy benchmark kicking around? Merlin Moncure <mmoncure@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox