pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedPostgresql Performance on an HP DL385 and SmartArray 642
70+ messages / 17 participants
[nested] [flat]
* Postgresql Performance on an HP DL385 and SmartArray 642
@ 2006-08-05 23:10 Steve Poe <steve.poe@gmail.com>
0 siblings, 2 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-05 23:10 UTC (permalink / raw)
To: pgsql-performance
I am do some consulting for an animal hospital in the Boston, MA area.
They wanted a new server to run their database on. The client wants
everything from one vendor, they wanted Dell initially, I'd advised
against it. I recommended a dual Opteron system from either Sun or HP.
They settled on a DL385 8GB of RAM with two disc U320 SCSI and a 6-disc
U320 SCSI array. I recommended they add a RAID adapter with at 128MB and
battery backup, they added a HP SmartArray 642 to connect to the drive
array in addition to the SmartArray 6i which came with the server.
Has anyone worked with server before. I've read the SmartArray 6i is a
poor performer, I wonder if the SmartArray 642 adapter would have the
same fate?
The database data is on the drive array(RAID10) and the pg_xlog is on
the internal RAID1 on the 6i controller. The results have been poor.
My guess is the controllers are garbage.
Thanks for any advice.
Steve Poe
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 01:10 Luke Lonergan <llonergan@greenplum.com>
parent: Steve Poe <steve.poe@gmail.com>
1 sibling, 4 replies; 70+ messages in thread
From: Luke Lonergan @ 2006-08-08 01:10 UTC (permalink / raw)
To: steve.poe@gmail.com; pgsql-performance
Steve,
On 8/5/06 4:10 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
> I am do some consulting for an animal hospital in the Boston, MA area.
> They wanted a new server to run their database on. The client wants
> everything from one vendor, they wanted Dell initially, I'd advised
> against it. I recommended a dual Opteron system from either Sun or HP.
> They settled on a DL385 8GB of RAM with two disc U320 SCSI and a 6-disc
> U320 SCSI array. I recommended they add a RAID adapter with at 128MB and
> battery backup, they added a HP SmartArray 642 to connect to the drive
> array in addition to the SmartArray 6i which came with the server.
>
> Has anyone worked with server before. I've read the SmartArray 6i is a
> poor performer, I wonder if the SmartArray 642 adapter would have the
> same fate?
>
> The database data is on the drive array(RAID10) and the pg_xlog is on
> the internal RAID1 on the 6i controller. The results have been poor.
>
> My guess is the controllers are garbage.
Can you run bonnie++ version 1.03a on the machine and report the results
here?
It could be OK if you have the latest Linux driver for cciss, someone has
reported good results to this list with the latest, bleeding edge version of
Linux (2.6.17).
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 01:46 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
3 siblings, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-08 01:46 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: pgsql-performance
Luke,
I'll do that then post the results. I ran zcav on it (default settlings) on
the disc array formatted XFS and its peak MB/s was around 85-90. I am using
kernel 2.6.17.7. mounting the disc array with noatime, nodiratime.
Thanks for your feedback.
Steve
On 8/7/06, Luke Lonergan <llonergan@greenplum.com> wrote:
>
> Steve,
>
> On 8/5/06 4:10 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
>
> > I am do some consulting for an animal hospital in the Boston, MA area.
> > They wanted a new server to run their database on. The client wants
> > everything from one vendor, they wanted Dell initially, I'd advised
> > against it. I recommended a dual Opteron system from either Sun or HP.
> > They settled on a DL385 8GB of RAM with two disc U320 SCSI and a 6-disc
> > U320 SCSI array. I recommended they add a RAID adapter with at 128MB and
> > battery backup, they added a HP SmartArray 642 to connect to the drive
> > array in addition to the SmartArray 6i which came with the server.
> >
> > Has anyone worked with server before. I've read the SmartArray 6i is a
> > poor performer, I wonder if the SmartArray 642 adapter would have the
> > same fate?
> >
> > The database data is on the drive array(RAID10) and the pg_xlog is on
> > the internal RAID1 on the 6i controller. The results have been poor.
> >
> > My guess is the controllers are garbage.
>
> Can you run bonnie++ version 1.03a on the machine and report the results
> here?
>
> It could be OK if you have the latest Linux driver for cciss, someone has
> reported good results to this list with the latest, bleeding edge version
> of
> Linux (2.6.17).
>
> - Luke
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 02:19 Joshua D. Drake <jd@commandprompt.com>
parent: Luke Lonergan <llonergan@greenplum.com>
3 siblings, 1 reply; 70+ messages in thread
From: Joshua D. Drake @ 2006-08-08 02:19 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: steve.poe@gmail.com; pgsql-performance
>> The database data is on the drive array(RAID10) and the pg_xlog is on
>> the internal RAID1 on the 6i controller. The results have been poor.
I have heard that the 6i was actually decent but to avoid the 5i.
Joshua D. Drake
>>
>> My guess is the controllers are garbage.
>
> Can you run bonnie++ version 1.03a on the machine and report the results
> here?
>
> It could be OK if you have the latest Linux driver for cciss, someone has
> reported good results to this list with the latest, bleeding edge version of
> Linux (2.6.17).
>
> - Luke
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 02:28 Steve Poe <steve.poe@gmail.com>
parent: Joshua D. Drake <jd@commandprompt.com>
0 siblings, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-08 02:28 UTC (permalink / raw)
To: Joshua D. Drake <jd@commandprompt.com>; +Cc: Luke Lonergan <llonergan@greenplum.com>; pgsql-performance
There is 64MB on the 6i and 192MB on the 642 controller. I wish the
controllers had a "wrieback" enable option like the LSI MegaRAID adapters
have. I have tried splitting the cache accelerator 25/75 75/25 0/100 100/0
but the results really did not improve.
Steve
On 8/7/06, Joshua D. Drake <jd@commandprompt.com> wrote:
>
>
> >> The database data is on the drive array(RAID10) and the pg_xlog is on
> >> the internal RAID1 on the 6i controller. The results have been poor.
>
> I have heard that the 6i was actually decent but to avoid the 5i.
>
> Joshua D. Drake
>
>
> >>
> >> My guess is the controllers are garbage.
> >
> > Can you run bonnie++ version 1.03a on the machine and report the results
> > here?
> >
> > It could be OK if you have the latest Linux driver for cciss, someone
> has
> > reported good results to this list with the latest, bleeding edge
> version of
> > Linux (2.6.17).
> >
> > - Luke
> >
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 9: In versions below 8.0, the planner will ignore your desire to
> > choose an index scan if your joining column's datatypes do not
> > match
> >
>
>
> --
>
> === The PostgreSQL Company: Command Prompt, Inc. ===
> Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
> Providing the most comprehensive PostgreSQL solutions since 1997
> http://www.commandprompt.com/
>
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 04:13 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
3 siblings, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-08 04:13 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: pgsql-performance
Luke,
Here are the results of two runs of 16GB file tests on XFS.
scsi disc array
xfs ,16G,81024,99,153016,24,73422,10,82092,97,243210,17,1043.1,0,16,3172,7,+++++,+++,2957,9,3197,10,+++++,+++,2484,8
scsi disc array
xfs ,16G,83320,99,155641,25,73662,10,81756,96,243352,18,1029.1,0,16,3119,10,+++++,+++,2789,7,3263,11,+++++,+++,2014,6
Thanks.
Steve
> Can you run bonnie++ version 1.03a on the machine and report the results
> here?
>
> It could be OK if you have the latest Linux driver for cciss, someone has
> reported good results to this list with the latest, bleeding edge version of
> Linux (2.6.17).
>
> - Luke
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 06:40 Alex Turner <armtuk@gmail.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 1 reply; 70+ messages in thread
From: Alex Turner @ 2006-08-08 06:40 UTC (permalink / raw)
To: steve.poe@gmail.com; +Cc: Luke Lonergan <llonergan@greenplum.com>; pgsql-performance
These number are pretty darn good for a four disk RAID 10, pretty close to
perfect infact. Nice advert for the 642 - I guess we have a Hardware RAID
controller than will read indpendently from mirrors.
Alex
On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
>
> Luke,
>
> Here are the results of two runs of 16GB file tests on XFS.
>
> scsi disc array
> xfs ,16G,81024,99,153016,24,73422,10,82092,97,243210,17,1043.1
> ,0,16,3172,7,+++++,+++,2957,9,3197,10,+++++,+++,2484,8
> scsi disc array
> xfs ,16G,83320,99,155641,25,73662,10,81756,96,243352,18,1029.1
> ,0,16,3119,10,+++++,+++,2789,7,3263,11,+++++,+++,2014,6
>
> Thanks.
>
> Steve
>
>
>
> > Can you run bonnie++ version 1.03a on the machine and report the results
> > here?
> >
> > It could be OK if you have the latest Linux driver for cciss, someone
> has
> > reported good results to this list with the latest, bleeding edge
> version of
> > Linux (2.6.17).
> >
> > - Luke
> >
>
>
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 9: In versions below 8.0, the planner will ignore your desire to
> > choose an index scan if your joining column's datatypes do not
> > match
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and SmartArray 642
@ 2006-08-08 12:20 Jeff Trout <threshar@torgo.978.org>
parent: Steve Poe <steve.poe@gmail.com>
1 sibling, 0 replies; 70+ messages in thread
From: Jeff Trout @ 2006-08-08 12:20 UTC (permalink / raw)
To: steve.poe@gmail.com; +Cc: pgsql-performance
On Aug 5, 2006, at 7:10 PM, Steve Poe wrote:
>
> Has anyone worked with server before. I've read the SmartArray 6i is a
> poor performer, I wonder if the SmartArray 642 adapter would have the
> same fate?
>
My newest db is a DL385, 6 disks. It runs very nicely. I have no
issues with the 6i controller.
If you look in the pgsql-performance archives a week or two ago
you'll see a similar thread to
this one - in fact, it is also about a dl385 (but he had a 5i
controller)
--
Jeff Trout <jeff@jefftrout.com>
http://www.dellsmartexitin.com/
http://www.stuarthamm.net/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 13:57 Steve Poe <steve.poe@gmail.com>
parent: Alex Turner <armtuk@gmail.com>
0 siblings, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-08 13:57 UTC (permalink / raw)
To: Alex Turner <armtuk@gmail.com>; +Cc: Luke Lonergan <llonergan@greenplum.com>; pgsql-performance
Alex,
Maybe I mis-stated, this is a 6-disk array.
Steve
On 8/7/06, Alex Turner <armtuk@gmail.com> wrote:
>
> These number are pretty darn good for a four disk RAID 10, pretty close to
> perfect infact. Nice advert for the 642 - I guess we have a Hardware RAID
> controller than will read indpendently from mirrors.
>
> Alex
>
> On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
>
> > Luke,
>
> Here are the results of two runs of 16GB file tests on XFS.
>
> scsi disc array
> xfs ,16G,81024,99,153016,24,73422,10,82092,97,243210,17,1043.1
> ,0,16,3172,7,+++++,+++,2957,9,3197,10,+++++,+++,2484,8
> scsi disc array
> xfs ,16G,83320,99,155641,25,73662,10,81756,96,243352,18,1029.1
> ,0,16,3119,10,+++++,+++,2789,7,3263,11,+++++,+++,2014,6
>
> Thanks.
>
> Steve
>
>
>
> > Can you run bonnie++ version 1.03a on the machine and report the results
>
> > here?
> >
> > It could be OK if you have the latest Linux driver for cciss, someone
> has
> > reported good results to this list with the latest, bleeding edge
> version of
> > Linux (2.6.17).
> >
> > - Luke
> >
>
>
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 9: In versions below 8.0, the planner will ignore your desire to
> > choose an index scan if your joining column's datatypes do not
> > match
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 14:36 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-08 14:36 UTC (permalink / raw)
To: Alex Turner <armtuk@gmail.com>; steve.poe@gmail.com; +Cc: pgsql-performance
I agree, I think these say you are getting 240MB/s sequential reads and 1000 seeks per second.
That's pretty much the best you'd expect.
- Luke
Sent from my GoodLink synchronized handheld (www.good.com)
-----Original Message-----
From: Alex Turner [mailto:armtuk@gmail.com]
Sent: Tuesday, August 08, 2006 02:40 AM Eastern Standard Time
To: steve.poe@gmail.com
Cc: Luke Lonergan; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Postgresql Performance on an HP DL385 and
These number are pretty darn good for a four disk RAID 10, pretty close to
perfect infact. Nice advert for the 642 - I guess we have a Hardware RAID
controller than will read indpendently from mirrors.
Alex
On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
>
> Luke,
>
> Here are the results of two runs of 16GB file tests on XFS.
>
> scsi disc array
> xfs ,16G,81024,99,153016,24,73422,10,82092,97,243210,17,1043.1
> ,0,16,3172,7,+++++,+++,2957,9,3197,10,+++++,+++,2484,8
> scsi disc array
> xfs ,16G,83320,99,155641,25,73662,10,81756,96,243352,18,1029.1
> ,0,16,3119,10,+++++,+++,2789,7,3263,11,+++++,+++,2014,6
>
> Thanks.
>
> Steve
>
>
>
> > Can you run bonnie++ version 1.03a on the machine and report the results
> > here?
> >
> > It could be OK if you have the latest Linux driver for cciss, someone
> has
> > reported good results to this list with the latest, bleeding edge
> version of
> > Linux (2.6.17).
> >
> > - Luke
> >
>
>
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 9: In versions below 8.0, the planner will ignore your desire to
> > choose an index scan if your joining column's datatypes do not
> > match
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 15:01 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-08 15:01 UTC (permalink / raw)
To: Luke Lonergan <LLonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Luke,
Thanks for the feedback. I use the same database test that I've run a Sun
dual Opteron with 4Gb RAM and (2) four disk arrays in RAID10. The sun box
with one disc on an LSI MegaRAID 2-channel adapter outperforms this HP box.
I though I was doing something wrong or there is something wrong with the
box.
Steve
On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
>
> I agree, I think these say you are getting 240MB/s sequential reads and
> 1000 seeks per second.
>
> That's pretty much the best you'd expect.
>
> - Luke
>
> Sent from my GoodLink synchronized handheld (www.good.com)
>
>
> -----Original Message-----
> From: Alex Turner [mailto:armtuk@gmail.com]
> Sent: Tuesday, August 08, 2006 02:40 AM Eastern Standard Time
> To: steve.poe@gmail.com
> Cc: Luke Lonergan; pgsql-performance@postgresql.org
> Subject: Re: [PERFORM] Postgresql Performance on an HP DL385 and
>
> These number are pretty darn good for a four disk RAID 10, pretty close to
> perfect infact. Nice advert for the 642 - I guess we have a Hardware RAID
> controller than will read indpendently from mirrors.
>
> Alex
>
> On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
> >
> > Luke,
> >
> > Here are the results of two runs of 16GB file tests on XFS.
> >
> > scsi disc array
> > xfs ,16G,81024,99,153016,24,73422,10,82092,97,243210,17,1043.1
> > ,0,16,3172,7,+++++,+++,2957,9,3197,10,+++++,+++,2484,8
> > scsi disc array
> > xfs ,16G,83320,99,155641,25,73662,10,81756,96,243352,18,1029.1
> > ,0,16,3119,10,+++++,+++,2789,7,3263,11,+++++,+++,2014,6
> >
> > Thanks.
> >
> > Steve
> >
> >
> >
> > > Can you run bonnie++ version 1.03a on the machine and report the
> results
> > > here?
> > >
> > > It could be OK if you have the latest Linux driver for cciss, someone
> > has
> > > reported good results to this list with the latest, bleeding edge
> > version of
> > > Linux (2.6.17).
> > >
> > > - Luke
> > >
> >
> >
> > >
> > >
> > > ---------------------------(end of
> broadcast)---------------------------
> > > TIP 9: In versions below 8.0, the planner will ignore your desire to
> > > choose an index scan if your joining column's datatypes do not
> > > match
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 3: Have you checked our extensive FAQ?
> >
> > http://www.postgresql.org/docs/faq
> >
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 16:22 Luke Lonergan <llonergan@greenplum.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-08 16:22 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Steve,
On 8/8/06 8:01 AM, "Steve Poe" <steve.poe@gmail.com> wrote:
> Thanks for the feedback. I use the same database test that I've run a Sun
> dual Opteron with 4Gb RAM and (2) four disk arrays in RAID10. The sun box with
> one disc on an LSI MegaRAID 2-channel adapter outperforms this HP box. I
> though I was doing something wrong or there is something wrong with the box.
Given the circumstances (benchmarked I/O is great, comparable perf on
another box with single disk is better), seems that one of:
1) something wrong with the CPU/memory on the box
2) something with the OS version / kernel
3) something with the postgres configuration
Can you post the database benchmark results?
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-08 16:57 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-08 16:57 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Luke,
Here's some background:
I use Pg 7.4.13 (I've tested as far back as 7.4.8). I use an 8GB data with a
program called odbc-bench. I run an 18 minute test. With each run, HP box
excluded, I unmount the discs involved, reformat, un-tar the backup of
PGDATA and pg_xlog back on the discs, start-up Postgresql, then run the
odbc-bench.
On the Sun box, I've benchmarked an average of 3 to 4 runs with each disc
(up to 8) in succession in RAID0, RAID5, and RAID10 where applicable. I've
done with with pg_xlog on the same discs as PGDATA and separately, so I've
felt like I had a good understanding of how the performance works. I've
notice performance seems to level off at around 6 discs with another 10-15%
with two more discs.
When I run odbc-bench, I also run vmstat in the background (through a python
script) which averages/summarzies the high/low/average of each category for
each minute then a final summary after the run.
On the Sun box, with 4 discs (RAID10) to one channel on the LSI RAID card, I
see an average TPS around 70. If I ran this off of one disc, I see an
average TPS of 32.
on the HP box, with 6-discs in RAID10 and 1 spare. I see a TPS of 34. I
don't have my vmstat reports with me, but I recall the CPU utilitization on
the HP was about 50% higher. I need to check on this.
Steve
On 8/8/06, Luke Lonergan <llonergan@greenplum.com> wrote:
>
> Steve,
>
> On 8/8/06 8:01 AM, "Steve Poe" <steve.poe@gmail.com> wrote:
>
> > Thanks for the feedback. I use the same database test that I've run a
> Sun
> > dual Opteron with 4Gb RAM and (2) four disk arrays in RAID10. The sun
> box with
> > one disc on an LSI MegaRAID 2-channel adapter outperforms this HP box. I
> > though I was doing something wrong or there is something wrong with the
> box.
>
> Given the circumstances (benchmarked I/O is great, comparable perf on
> another box with single disk is better), seems that one of:
> 1) something wrong with the CPU/memory on the box
> 2) something with the OS version / kernel
> 3) something with the postgres configuration
>
> Can you post the database benchmark results?
>
> - Luke
>
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 01:55 Luke Lonergan <llonergan@greenplum.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 2 replies; 70+ messages in thread
From: Luke Lonergan @ 2006-08-09 01:55 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Steve,
On 8/8/06 9:57 AM, "Steve Poe" <steve.poe@gmail.com> wrote:
> On the Sun box, with 4 discs (RAID10) to one channel on the LSI RAID card, I
> see an average TPS around 70. If I ran this off of one disc, I see an average
> TPS of 32.
>
> on the HP box, with 6-discs in RAID10 and 1 spare. I see a TPS of 34. I don't
> have my vmstat reports with me, but I recall the CPU utilitization on the HP
> was about 50% higher. I need to check on this.
Sounds like there are a few moving parts here, one of which is the ODBC
driver.
First - using 7.4.x postgres is a big variable - not much experience on this
list with 7.4.x anymore.
What OS versions are on the two machines?
What is the network configuration of each - is a caching DNS server
available to each? What are the contents of /etc/resolv.conf?
Have you run "top" on the machines while the benchmark is running? What is
the top running process, what is it doing (RSS, swap, I/O wait, etc)?
Are any of the disks not healthy? Do you see any I/O errors in dmesg?
Note that tarring up the database directory and untarring it actually
changes the block layout of the files on the disk from what the database
might have done when it was created. When you create a tar archive of the
files in the DB directory, their contents will be packed in file name order
in the tar archive and unpacked that way as well. By comparison, the
ordering when the database lays them on disk might have been quite
different. This doesn't impact the problem you describe as you are
unpacking the tar file on both machines to start the process (right?).
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 04:56 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
1 sibling, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 04:56 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
>
> >Sounds like there are a few moving parts here, one of which is the ODBC
> >driver.
Yes, I need to use it since my clients use it for their veterinary
application.
>First - using 7.4.x postgres is a big variable - not much experience on
> this
> >list with 7.4.x anymore.
Like the previous, we have to use it since the manufacturer/vendor uses a
4GL language which only supports Postgresql 7.4.x
>What OS versions are on the two machines?
Centos 4.3 x84_64 on both boxes.
>What is the network configuration of each - is a caching DNS server
> >available to each? What are the contents of /etc/resolv.conf?
The database is configured for the local/loopback on 127.0.0.1. This is my
local network. No DNS.
>Have you run "top" on the machines while the benchmark is running? What is
> >the top running process, what is it doing (RSS, swap, I/O wait, etc)?
I am not running top, but here's an average per second for the 20-25min
run from vmstat presented in a high/peak, low and median
Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10 LSI MegaRAID
128MB). This is after 8 runs.
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,swapd,128,128,128
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,free,21596,21050,21327
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,buffers,1171,174,595
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,cache,3514368,3467427,3495081
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,bi,97276,1720,31745
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,bo,9209,832,4674
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,in,25906,23204,24115
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,cs,49849,46035,47617
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
Average TPS is 75
HP box with 8GB RAM. six disc array RAID10 on SmartArray 642 with 192MB RAM.
After 8 runs, I see:
intown-vetstar-amd64,08/09/06,Tuesday,23,r,0,0,0
intown-vetstar-amd64,08/09/06,Tuesday,23,b,2,0,0
intown-vetstar-amd64,08/09/06,Tuesday,23,swapd,0,0,0
intown-vetstar-amd64,08/09/06,Tuesday,23,free,33760,16501,17931
intown-vetstar-amd64,08/09/06,Tuesday,23,buffers,1578,673,1179
intown-vetstar-amd64,08/09/06,Tuesday,23,cache,7881745,7867700,7876327
intown-vetstar-amd64,08/09/06,Tuesday,23,bi,66536,0,4480
intown-vetstar-amd64,08/09/06,Tuesday,23,bo,5991,2,2806
intown-vetstar-amd64,08/09/06,Tuesday,23,in,1624,260,573
intown-vetstar-amd64,08/09/06,Tuesday,23,cs,2342,17,1464
intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
Average TPS is 31.
>Are any of the disks not healthy? Do you see any I/O errors in dmesg?
I don't know. I do the following message:
"PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for subordinate bus"
Otherwise, no disc error messages.
Note that tarring up the database directory and untarring it actually
> changes the block layout of the files on the disk from what the database
> might have done when it was created. When you create a tar archive of the
> files in the DB directory, their contents will be packed in file name
> order
> in the tar archive and unpacked that way as well. By comparison, the
> ordering when the database lays them on disk might have been quite
> different. This doesn't impact the problem you describe as you are
> unpacking the tar file on both machines to start the process (right?).
Yes, I am running this on both machines with the same RPMs of Postgresql and
same conf files.
Also, just for this testing, I am not unmounting, formatting, untaring. I am
doing it once than running the series of tests (usually 10 runs).
Thanks again for your time. If you're in the SF area, I'll owe you lunch and
> beer.
Steve
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 05:22 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
1 sibling, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 05:22 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
> Are any of the disks not healthy? Do you see any I/O errors in dmesg?
Luke,
In my vmstat report, I it is an average per minute not per-second. Also,
I found that in the first minute of the very first run, the HP's "bi"
value hits a high of 221184 then it tanks after that.
Steve
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 05:22 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-09 05:22 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Steve,
> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> LSI MegaRAID 128MB). This is after 8 runs.
>
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
>
> Average TPS is 75
>
> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> with 192MB RAM. After 8 runs, I see:
>
> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
>
> Average TPS is 31.
Note that the I/O wait (wa) on the HP box high, low and average are all
*much* higher than on the Sun box. The average I/O wait was 50% of one
CPU, which is huge. By comparison there was virtually no I/O wait on
the Sun machine.
This is indicating that your HP machine is indeed I/O bound and
furthermore is tying up a PG process waiting for the disk to return.
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 05:24 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 0 replies; 70+ messages in thread
From: Luke Lonergan @ 2006-08-09 05:24 UTC (permalink / raw)
To: steve.poe@gmail.com; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Steve,
> > Are any of the disks not healthy? Do you see any I/O
> errors in dmesg?
>
> In my vmstat report, I it is an average per minute not
> per-second. Also, I found that in the first minute of the
> very first run, the HP's "bi"
> value hits a high of 221184 then it tanks after that.
Based on the difference in I/O wait on the two machines, there's
definitely something up with the disk subsystem on the HP - try checking
the BIOS on the HP 642 - I bet you'll find a disk error lurking there.
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 05:45 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 2 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 05:45 UTC (permalink / raw)
To: Luke Lonergan <LLonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Luke,
I thought so. In my test, I tried to be fair/equal since my Sun box has two
4-disc arrays each on their own channel. So, I just used one of them which
should be a little slower than the 6-disc with 192MB cache.
Incidently, the two internal SCSI drives, which are on the 6i adapter,
generated a TPS of 18.
I thought this server would impressive from notes I've read in the group.
This is why I thought I might be doing something wrong. I stumped which way
to take this. There is no obvious fault but something isn't right.
Steve
On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
>
> Steve,
>
> > Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > LSI MegaRAID 128MB). This is after 8 runs.
> >
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> >
> > Average TPS is 75
> >
> > HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > with 192MB RAM. After 8 runs, I see:
> >
> > intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> >
> > Average TPS is 31.
>
> Note that the I/O wait (wa) on the HP box high, low and average are all
> *much* higher than on the Sun box. The average I/O wait was 50% of one
> CPU, which is huge. By comparison there was virtually no I/O wait on
> the Sun machine.
>
> This is indicating that your HP machine is indeed I/O bound and
> furthermore is tying up a PG process waiting for the disk to return.
>
> - Luke
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 06:33 Steve Poe <steve.poe@gmail.com>
parent: Steve Poe <steve.poe@gmail.com>
1 sibling, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 06:33 UTC (permalink / raw)
To: Luke Lonergan <LLonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Luke,
I check dmesg one more time and I found this regarding the cciss driver:
Filesystem "cciss/c1d0p1": Disabling barriers, not supported by the
underlying device.
Don't know if it means anything, but thought I'd mention it.
Steve
On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
>
> Luke,
>
> I thought so. In my test, I tried to be fair/equal since my Sun box has
> two 4-disc arrays each on their own channel. So, I just used one of them
> which should be a little slower than the 6-disc with 192MB cache.
>
> Incidently, the two internal SCSI drives, which are on the 6i adapter,
> generated a TPS of 18.
>
> I thought this server would impressive from notes I've read in the group.
> This is why I thought I might be doing something wrong. I stumped which way
> to take this. There is no obvious fault but something isn't right.
>
> Steve
>
>
> On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> >
> > Steve,
> >
> > > Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > > LSI MegaRAID 128MB). This is after 8 runs.
> > >
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >
> > > Average TPS is 75
> > >
> > > HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > > with 192MB RAM. After 8 runs, I see:
> > >
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >
> > > Average TPS is 31.
> >
> > Note that the I/O wait (wa) on the HP box high, low and average are all
> > *much* higher than on the Sun box. The average I/O wait was 50% of one
> > CPU, which is huge. By comparison there was virtually no I/O wait on
> > the Sun machine.
> >
> > This is indicating that your HP machine is indeed I/O bound and
> > furthermore is tying up a PG process waiting for the disk to return.
> >
> > - Luke
> >
> >
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 06:35 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-09 06:35 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Steve,
At the end of the day it seems that you've got a support issue with the
SmartArray RAID adapter from HP.
Last I tried that I found that they don't write the cciss driver, don't
test it for performance on Linux and don't make any claims about it's
performance on Linux.
That said - can you contact them through HP tech support and report back
to this list what you find out?
- Luke
> -----Original Message-----
> From: Steve Poe [mailto:steve.poe@gmail.com]
> Sent: Tuesday, August 08, 2006 11:33 PM
> To: Luke Lonergan
> Cc: Alex Turner; pgsql-performance@postgresql.org
> Subject: Re: [PERFORM] Postgresql Performance on an HP DL385 and
>
> Luke,
>
> I check dmesg one more time and I found this regarding the
> cciss driver:
>
> Filesystem "cciss/c1d0p1": Disabling barriers, not supported
> by the underlying device.
>
> Don't know if it means anything, but thought I'd mention it.
>
> Steve
>
>
> On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
>
> Luke,
>
> I thought so. In my test, I tried to be fair/equal
> since my Sun box has two 4-disc arrays each on their own
> channel. So, I just used one of them which should be a little
> slower than the 6-disc with 192MB cache.
>
> Incidently, the two internal SCSI drives, which are on
> the 6i adapter, generated a TPS of 18.
>
> I thought this server would impressive from notes I've
> read in the group. This is why I thought I might be doing
> something wrong. I stumped which way to take this. There is
> no obvious fault but something isn't right.
>
>
> Steve
>
>
>
> On 8/8/06, Luke Lonergan < LLonergan@greenplum.com
> <mailto:LLonergan@greenplum.com> > wrote:
>
> Steve,
>
> > Sun box with 4-disc array (4GB RAM. 4 167GB
> 10K SCSI RAID10
> > LSI MegaRAID 128MB). This is after 8 runs.
> >
> >
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> >
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> >
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> >
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> >
> > Average TPS is 75
> >
> > HP box with 8GB RAM. six disc array RAID10 on
> SmartArray 642
> > with 192MB RAM. After 8 runs, I see:
> >
> > intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> >
> > Average TPS is 31.
>
> Note that the I/O wait (wa) on the HP box high,
> low and average are all
> *much* higher than on the Sun box. The average
> I/O wait was 50% of one
> CPU, which is huge. By comparison there was
> virtually no I/O wait on
> the Sun machine.
>
> This is indicating that your HP machine is
> indeed I/O bound and
> furthermore is tying up a PG process waiting
> for the disk to return.
>
> - Luke
>
>
>
>
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 06:47 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 06:47 UTC (permalink / raw)
To: Luke Lonergan <LLonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Luke,
I will do that. If it is the general impression that this server should
perform well with Postgresql, Are the RAID cards, the 6i and 642 sufficient
to your knowledge? I am wondering if it is the disc array itself.
Steve
On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
>
> Steve,
>
> At the end of the day it seems that you've got a support issue with the
> SmartArray RAID adapter from HP.
>
> Last I tried that I found that they don't write the cciss driver, don't
> test it for performance on Linux and don't make any claims about it's
> performance on Linux.
>
> That said - can you contact them through HP tech support and report back
> to this list what you find out?
>
> - Luke
> > -----Original Message-----
> > From: Steve Poe [mailto:steve.poe@gmail.com]
> > Sent: Tuesday, August 08, 2006 11:33 PM
> > To: Luke Lonergan
> > Cc: Alex Turner; pgsql-performance@postgresql.org
> > Subject: Re: [PERFORM] Postgresql Performance on an HP DL385 and
> >
> > Luke,
> >
> > I check dmesg one more time and I found this regarding the
> > cciss driver:
> >
> > Filesystem "cciss/c1d0p1": Disabling barriers, not supported
> > by the underlying device.
> >
> > Don't know if it means anything, but thought I'd mention it.
> >
> > Steve
> >
> >
> > On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
> >
> > Luke,
> >
> > I thought so. In my test, I tried to be fair/equal
> > since my Sun box has two 4-disc arrays each on their own
> > channel. So, I just used one of them which should be a little
> > slower than the 6-disc with 192MB cache.
> >
> > Incidently, the two internal SCSI drives, which are on
> > the 6i adapter, generated a TPS of 18.
> >
> > I thought this server would impressive from notes I've
> > read in the group. This is why I thought I might be doing
> > something wrong. I stumped which way to take this. There is
> > no obvious fault but something isn't right.
> >
> >
> > Steve
> >
> >
> >
> > On 8/8/06, Luke Lonergan < LLonergan@greenplum.com
> > <mailto:LLonergan@greenplum.com> > wrote:
> >
> > Steve,
> >
> > > Sun box with 4-disc array (4GB RAM. 4 167GB
> > 10K SCSI RAID10
> > > LSI MegaRAID 128MB). This is after 8 runs.
> > >
> > >
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > >
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > >
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > >
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >
> > > Average TPS is 75
> > >
> > > HP box with 8GB RAM. six disc array RAID10 on
> > SmartArray 642
> > > with 192MB RAM. After 8 runs, I see:
> > >
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >
> > > Average TPS is 31.
> >
> > Note that the I/O wait (wa) on the HP box high,
> > low and average are all
> > *much* higher than on the Sun box. The average
> > I/O wait was 50% of one
> > CPU, which is huge. By comparison there was
> > virtually no I/O wait on
> > the Sun machine.
> >
> > This is indicating that your HP machine is
> > indeed I/O bound and
> > furthermore is tying up a PG process waiting
> > for the disk to return.
> >
> > - Luke
> >
> >
> >
> >
> >
> >
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 06:50 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-09 06:50 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Steve,
> I will do that. If it is the general impression that this
> server should perform well with Postgresql, Are the RAID
> cards, the 6i and 642 sufficient to your knowledge? I am
> wondering if it is the disc array itself.
I think that is the question to be answered by HP support. Ask them for
technical support for the Linux driver for the 6i and 642. If they
offer support, they should quickly figure out what the problem is. If
they don't provide support, I would send the server back to them if you
can and buy a Sun server with an LSI RAID adapter.
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 06:56 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 06:56 UTC (permalink / raw)
To: Luke Lonergan <LLonergan@greenplum.com>; +Cc: Alex Turner <armtuk@gmail.com>; pgsql-performance
Luke,
I hope so. I'll keep you and the list up-to-date as I learn more.
Steve
On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
>
> Steve,
>
> > I will do that. If it is the general impression that this
> > server should perform well with Postgresql, Are the RAID
> > cards, the 6i and 642 sufficient to your knowledge? I am
> > wondering if it is the disc array itself.
>
> I think that is the question to be answered by HP support. Ask them for
> technical support for the Linux driver for the 6i and 642. If they
> offer support, they should quickly figure out what the problem is. If
> they don't provide support, I would send the server back to them if you
> can and buy a Sun server with an LSI RAID adapter.
>
> - Luke
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 21:05 Jim C. Nasby <jnasby@pervasive.com>
parent: Steve Poe <steve.poe@gmail.com>
1 sibling, 2 replies; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-09 21:05 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Luke Lonergan <LLonergan@greenplum.com>; Alex Turner <armtuk@gmail.com>; pgsql-performance
On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote:
> Luke,
>
> I thought so. In my test, I tried to be fair/equal since my Sun box has two
> 4-disc arrays each on their own channel. So, I just used one of them which
> should be a little slower than the 6-disc with 192MB cache.
>
> Incidently, the two internal SCSI drives, which are on the 6i adapter,
> generated a TPS of 18.
You should try putting pg_xlog on the 6 drive array with the data. My
(limited) experience with such a config is that on a good controller
with writeback caching enabled it won't hurt you, and if the internal
drives aren't caching writes it'll probably help you a lot.
> I thought this server would impressive from notes I've read in the group.
> This is why I thought I might be doing something wrong. I stumped which way
> to take this. There is no obvious fault but something isn't right.
>
> Steve
>
> On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> >
> >Steve,
> >
> >> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> >> LSI MegaRAID 128MB). This is after 8 runs.
> >>
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> >>
> >> Average TPS is 75
> >>
> >> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> >> with 192MB RAM. After 8 runs, I see:
> >>
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> >>
> >> Average TPS is 31.
> >
> >Note that the I/O wait (wa) on the HP box high, low and average are all
> >*much* higher than on the Sun box. The average I/O wait was 50% of one
> >CPU, which is huge. By comparison there was virtually no I/O wait on
> >the Sun machine.
> >
> >This is indicating that your HP machine is indeed I/O bound and
> >furthermore is tying up a PG process waiting for the disk to return.
> >
> >- Luke
> >
> >
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 21:11 Steve Poe <steve.poe@gmail.com>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-09 21:11 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: Luke Lonergan <LLonergan@greenplum.com>; Alex Turner <armtuk@gmail.com>; pgsql-performance
Jim,
I'll give it a try. However, I did not see anywhere in the BIOS
configuration of the 642 RAID adapter to enable writeback. It may have been
mislabled cache accelerator where you can give a percentage to read/write.
That aspect did not change the performance like the LSI MegaRAID adapter
does.
Steve
On 8/9/06, Jim C. Nasby <jnasby@pervasive.com> wrote:
>
> On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote:
> > Luke,
> >
> > I thought so. In my test, I tried to be fair/equal since my Sun box has
> two
> > 4-disc arrays each on their own channel. So, I just used one of them
> which
> > should be a little slower than the 6-disc with 192MB cache.
> >
> > Incidently, the two internal SCSI drives, which are on the 6i adapter,
> > generated a TPS of 18.
>
> You should try putting pg_xlog on the 6 drive array with the data. My
> (limited) experience with such a config is that on a good controller
> with writeback caching enabled it won't hurt you, and if the internal
> drives aren't caching writes it'll probably help you a lot.
>
> > I thought this server would impressive from notes I've read in the
> group.
> > This is why I thought I might be doing something wrong. I stumped which
> way
> > to take this. There is no obvious fault but something isn't right.
> >
> > Steve
> >
> > On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> > >
> > >Steve,
> > >
> > >> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > >> LSI MegaRAID 128MB). This is after 8 runs.
> > >>
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >>
> > >> Average TPS is 75
> > >>
> > >> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > >> with 192MB RAM. After 8 runs, I see:
> > >>
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >>
> > >> Average TPS is 31.
> > >
> > >Note that the I/O wait (wa) on the HP box high, low and average are all
> > >*much* higher than on the Sun box. The average I/O wait was 50% of one
> > >CPU, which is huge. By comparison there was virtually no I/O wait on
> > >the Sun machine.
> > >
> > >This is indicating that your HP machine is indeed I/O bound and
> > >furthermore is tying up a PG process waiting for the disk to return.
> > >
> > >- Luke
> > >
> > >
>
> --
> Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
> Pervasive Software http://pervasive.com work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 21:37 Scott Marlowe <smarlowe@g2switchworks.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 2 replies; 70+ messages in thread
From: Scott Marlowe @ 2006-08-09 21:37 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Jim C. Nasby <jnasby@pervasive.com>; Luke Lonergan <LLonergan@greenplum.com>; Alex Turner <armtuk@gmail.com>; pgsql-performance
On Wed, 2006-08-09 at 16:11, Steve Poe wrote:
> Jim,
>
> I'll give it a try. However, I did not see anywhere in the BIOS
> configuration of the 642 RAID adapter to enable writeback. It may have
> been mislabled cache accelerator where you can give a percentage to
> read/write. That aspect did not change the performance like the LSI
> MegaRAID adapter does.
Nope, that's not the same thing.
Does your raid controller have batter backed cache, or plain or regular
cache? write back is unsafe without battery backup.
The default is write through (i.e. the card waits for the data to get
written out before acking an fsync). In write back, the card's driver
writes the data to the bb cache, then returns on an fsync while the
cache gets written out at leisure. In the event of a loss of power, the
cache is flushed on restart.
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-09 23:47 Steve Poe <steve.poe@gmail.com>
parent: Scott Marlowe <smarlowe@g2switchworks.com>
1 sibling, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-09 23:47 UTC (permalink / raw)
To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: Jim C. Nasby <jnasby@pervasive.com>; Luke Lonergan <LLonergan@greenplum.com>; Alex Turner <armtuk@gmail.com>; pgsql-performance
I believe it does, I'll need to check.Thanks for the correction.
Steve
On 8/9/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
>
> On Wed, 2006-08-09 at 16:11, Steve Poe wrote:
> > Jim,
> >
> > I'll give it a try. However, I did not see anywhere in the BIOS
> > configuration of the 642 RAID adapter to enable writeback. It may have
> > been mislabled cache accelerator where you can give a percentage to
> > read/write. That aspect did not change the performance like the LSI
> > MegaRAID adapter does.
>
> Nope, that's not the same thing.
>
> Does your raid controller have batter backed cache, or plain or regular
> cache? write back is unsafe without battery backup.
>
> The default is write through (i.e. the card waits for the data to get
> written out before acking an fsync). In write back, the card's driver
> writes the data to the bb cache, then returns on an fsync while the
> cache gets written out at leisure. In the event of a loss of power, the
> cache is flushed on restart.
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-10 01:24 Steve Poe <steve.poe@gmail.com>
parent: Scott Marlowe <smarlowe@g2switchworks.com>
1 sibling, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-10 01:24 UTC (permalink / raw)
To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: Jim C. Nasby <jnasby@pervasive.com>; Luke Lonergan <LLonergan@greenplum.com>; Alex Turner <armtuk@gmail.com>; pgsql-performance
Scott,
Do you know how to activate the writeback on the RAID controller from HP?
Steve
On 8/9/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
>
> On Wed, 2006-08-09 at 16:11, Steve Poe wrote:
> > Jim,
> >
> > I'll give it a try. However, I did not see anywhere in the BIOS
> > configuration of the 642 RAID adapter to enable writeback. It may have
> > been mislabled cache accelerator where you can give a percentage to
> > read/write. That aspect did not change the performance like the LSI
> > MegaRAID adapter does.
>
> Nope, that's not the same thing.
>
> Does your raid controller have batter backed cache, or plain or regular
> cache? write back is unsafe without battery backup.
>
> The default is write through (i.e. the card waits for the data to get
> written out before acking an fsync). In write back, the card's driver
> writes the data to the bb cache, then returns on an fsync while the
> cache gets written out at leisure. In the event of a loss of power, the
> cache is flushed on restart.
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-10 03:29 Steve Poe <steve.poe@gmail.com>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-10 03:29 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: Luke Lonergan <LLonergan@greenplum.com>; Alex Turner <armtuk@gmail.com>; pgsql-performance
Jim,
I tried as you suggested and my performance dropped by 50%. I went from
a 32 TPS to 16. Oh well.
Steve
On Wed, 2006-08-09 at 16:05 -0500, Jim C. Nasby wrote:
> On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote:
> > Luke,
> >
> > I thought so. In my test, I tried to be fair/equal since my Sun box has two
> > 4-disc arrays each on their own channel. So, I just used one of them which
> > should be a little slower than the 6-disc with 192MB cache.
> >
> > Incidently, the two internal SCSI drives, which are on the 6i adapter,
> > generated a TPS of 18.
>
> You should try putting pg_xlog on the 6 drive array with the data. My
> (limited) experience with such a config is that on a good controller
> with writeback caching enabled it won't hurt you, and if the internal
> drives aren't caching writes it'll probably help you a lot.
>
> > I thought this server would impressive from notes I've read in the group.
> > This is why I thought I might be doing something wrong. I stumped which way
> > to take this. There is no obvious fault but something isn't right.
> >
> > Steve
> >
> > On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> > >
> > >Steve,
> > >
> > >> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > >> LSI MegaRAID 128MB). This is after 8 runs.
> > >>
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >>
> > >> Average TPS is 75
> > >>
> > >> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > >> with 192MB RAM. After 8 runs, I see:
> > >>
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >>
> > >> Average TPS is 31.
> > >
> > >Note that the I/O wait (wa) on the HP box high, low and average are all
> > >*much* higher than on the Sun box. The average I/O wait was 50% of one
> > >CPU, which is huge. By comparison there was virtually no I/O wait on
> > >the Sun machine.
> > >
> > >This is indicating that your HP machine is indeed I/O bound and
> > >furthermore is tying up a PG process waiting for the disk to return.
> > >
> > >- Luke
> > >
> > >
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-10 11:09 Michael Stone <mstone+postgres@mathom.us>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 2 replies; 70+ messages in thread
From: Michael Stone @ 2006-08-10 11:09 UTC (permalink / raw)
To: pgsql-performance
On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
>I tried as you suggested and my performance dropped by 50%. I went from
>a 32 TPS to 16. Oh well.
If you put data & xlog on the same array, put them on seperate
partitions, probably formatted differently (ext2 on xlog).
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-10 15:15 Luke Lonergan <llonergan@greenplum.com>
parent: Michael Stone <mstone+postgres@mathom.us>
1 sibling, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-10 15:15 UTC (permalink / raw)
To: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Mike,
On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
> On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
>> I tried as you suggested and my performance dropped by 50%. I went from
>> a 32 TPS to 16. Oh well.
>
> If you put data & xlog on the same array, put them on seperate
> partitions, probably formatted differently (ext2 on xlog).
If he's doing the same thing on both systems (Sun and HP) and the HP
performance is dramatically worse despite using more disks and having faster
CPUs and more RAM, ISTM the problem isn't the configuration.
Add to this the fact that the Sun machine is CPU bound while the HP is I/O
wait bound and I think the problem is the disk hardware or the driver
therein.
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-10 15:35 Scott Marlowe <smarlowe@g2switchworks.com>
parent: Luke Lonergan <llonergan@greenplum.com>
0 siblings, 2 replies; 70+ messages in thread
From: Scott Marlowe @ 2006-08-10 15:35 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote:
> Mike,
>
> On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
>
> > On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
> >> I tried as you suggested and my performance dropped by 50%. I went from
> >> a 32 TPS to 16. Oh well.
> >
> > If you put data & xlog on the same array, put them on seperate
> > partitions, probably formatted differently (ext2 on xlog).
>
> If he's doing the same thing on both systems (Sun and HP) and the HP
> performance is dramatically worse despite using more disks and having faster
> CPUs and more RAM, ISTM the problem isn't the configuration.
>
> Add to this the fact that the Sun machine is CPU bound while the HP is I/O
> wait bound and I think the problem is the disk hardware or the driver
> therein.
I agree. The problem here looks to be the RAID controller.
Steve, got access to a different RAID controller to test with?
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-10 15:47 Steve Poe <steve.poe@gmail.com>
parent: Scott Marlowe <smarlowe@g2switchworks.com>
1 sibling, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-10 15:47 UTC (permalink / raw)
To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: Luke Lonergan <llonergan@greenplum.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Scott,
I *could* rip out the LSI MegaRAID 2X from my Sun box. This belongs to me
for testing. but I don't know if it will fit in the DL385. Do they have
full-heigth/length slots? I've not worked on this type of box before. I was
thinking this is the next step. In the meantime, I've discovered their no
email support for them so I am hoping find a support contact through the
sales rep that this box was purchased from.
Steve
On 8/10/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
>
> On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote:
> > Mike,
> >
> > On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
> >
> > > On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
> > >> I tried as you suggested and my performance dropped by 50%. I went
> from
> > >> a 32 TPS to 16. Oh well.
> > >
> > > If you put data & xlog on the same array, put them on seperate
> > > partitions, probably formatted differently (ext2 on xlog).
> >
> > If he's doing the same thing on both systems (Sun and HP) and the HP
> > performance is dramatically worse despite using more disks and having
> faster
> > CPUs and more RAM, ISTM the problem isn't the configuration.
> >
> > Add to this the fact that the Sun machine is CPU bound while the HP is
> I/O
> > wait bound and I think the problem is the disk hardware or the driver
> > therein.
>
> I agree. The problem here looks to be the RAID controller.
>
> Steve, got access to a different RAID controller to test with?
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-14 15:38 Jim C. Nasby <jnasby@pervasive.com>
parent: Michael Stone <mstone+postgres@mathom.us>
1 sibling, 2 replies; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-14 15:38 UTC (permalink / raw)
To: pgsql-performance
On Thu, Aug 10, 2006 at 07:09:38AM -0400, Michael Stone wrote:
> On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
> >I tried as you suggested and my performance dropped by 50%. I went from
> >a 32 TPS to 16. Oh well.
>
> If you put data & xlog on the same array, put them on seperate
> partitions, probably formatted differently (ext2 on xlog).
Got any data to back that up?
The problem with seperate partitions is that it means more head movement
for the drives. If it's all one partition the pg_xlog data will tend to
be interspersed with the heap data, meaning less need for head
repositioning.
Of course, if ext2 provided enough of a speed improvement over ext3 with
data=writeback then it's possible that this would be a win, though if
the controller is good enough to make putting pg_xlog on the same array
as $PGDATA a win, I suspect it would make up for most filesystem
performance issues associated with pg_xlog as well.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-14 15:51 Steve Poe <steve.poe@gmail.com>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-14 15:51 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: pgsql-performance
Jim,
I have to say Michael is onto something here to my surprise. I partitioned
the RAID10 on the SmartArray 642 adapter into two parts, PGDATA formatted
with XFS and pg_xlog as ext2. Performance jumped up to median of 98 TPS. I
could reproduce the similar result with the LSI MegaRAID 2X adapter as well
as with my own 4-disc drive array.
The problem lies with the HP SmartArray 6i adapter and/or the internal SCSI
discs. Putting the pg_xlog on it kills the performance.
Steve
On 8/14/06, Jim C. Nasby <jnasby@pervasive.com> wrote:
>
> On Thu, Aug 10, 2006 at 07:09:38AM -0400, Michael Stone wrote:
> > On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
> > >I tried as you suggested and my performance dropped by 50%. I went from
> > >a 32 TPS to 16. Oh well.
> >
> > If you put data & xlog on the same array, put them on seperate
> > partitions, probably formatted differently (ext2 on xlog).
>
> Got any data to back that up?
>
> The problem with seperate partitions is that it means more head movement
> for the drives. If it's all one partition the pg_xlog data will tend to
> be interspersed with the heap data, meaning less need for head
> repositioning.
>
> Of course, if ext2 provided enough of a speed improvement over ext3 with
> data=writeback then it's possible that this would be a win, though if
> the controller is good enough to make putting pg_xlog on the same array
> as $PGDATA a win, I suspect it would make up for most filesystem
> performance issues associated with pg_xlog as well.
> --
> Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
> Pervasive Software http://pervasive.com work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-14 17:03 Michael Stone <mstone+postgres@mathom.us>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 1 reply; 70+ messages in thread
From: Michael Stone @ 2006-08-14 17:03 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: pgsql-performance
On Mon, Aug 14, 2006 at 10:38:41AM -0500, Jim C. Nasby wrote:
>Got any data to back that up?
yes. that I'm willing to dig out? no. :)
>The problem with seperate partitions is that it means more head movement
>for the drives. If it's all one partition the pg_xlog data will tend to
>be interspersed with the heap data, meaning less need for head
>repositioning.
The pg_xlog files will tend to be created up at the front of the disk
and just sit there. Any affect the positioning has one way or the other
isn't going to be measurable/repeatable. With a write cache for pg_xlog
the positioning isn't really going to matter anyway, since you don't
have to wait for a seek to do the write.
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-14 17:05 Jim C. Nasby <jnasby@pervasive.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 1 reply; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-14 17:05 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: pgsql-performance
On Mon, Aug 14, 2006 at 08:51:09AM -0700, Steve Poe wrote:
> Jim,
>
> I have to say Michael is onto something here to my surprise. I partitioned
> the RAID10 on the SmartArray 642 adapter into two parts, PGDATA formatted
> with XFS and pg_xlog as ext2. Performance jumped up to median of 98 TPS. I
> could reproduce the similar result with the LSI MegaRAID 2X adapter as well
> as with my own 4-disc drive array.
>
> The problem lies with the HP SmartArray 6i adapter and/or the internal SCSI
> discs. Putting the pg_xlog on it kills the performance.
Wow, interesting. IIRC, XFS is lower performing than ext3, so if your
previous tests were done with XFS, that might be part of it. But without
a doubt, if you don't have a good raid controller you don't want to try
combining pg_xlog with PGDATA.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-14 17:09 Michael Stone <mstone+postgres@mathom.us>
parent: Jim C. Nasby <jnasby@pervasive.com>
0 siblings, 1 reply; 70+ messages in thread
From: Michael Stone @ 2006-08-14 17:09 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: Steve Poe <steve.poe@gmail.com>; pgsql-performance
On Mon, Aug 14, 2006 at 12:05:46PM -0500, Jim C. Nasby wrote:
>Wow, interesting. IIRC, XFS is lower performing than ext3,
For xlog, maybe. For data, no. Both are definately slower than ext2 for
xlog, which is another reason to have xlog on a small filesystem which
doesn't need metadata journalling.
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 16:25 Jim C. Nasby <jnasby@pervasive.com>
parent: Michael Stone <mstone+postgres@mathom.us>
0 siblings, 2 replies; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-15 16:25 UTC (permalink / raw)
To: pgsql-performance
On Mon, Aug 14, 2006 at 01:03:41PM -0400, Michael Stone wrote:
> On Mon, Aug 14, 2006 at 10:38:41AM -0500, Jim C. Nasby wrote:
> >Got any data to back that up?
>
> yes. that I'm willing to dig out? no. :)
Well, I'm not digging hard numbers out either, so that's fair. :) But it
would be very handy if people posted results from any testing they're
doing as part of setting up new hardware. Actually, a wiki would
probably be ideal for this...
> >The problem with seperate partitions is that it means more head movement
> >for the drives. If it's all one partition the pg_xlog data will tend to
> >be interspersed with the heap data, meaning less need for head
> >repositioning.
>
> The pg_xlog files will tend to be created up at the front of the disk
> and just sit there. Any affect the positioning has one way or the other
> isn't going to be measurable/repeatable. With a write cache for pg_xlog
> the positioning isn't really going to matter anyway, since you don't
> have to wait for a seek to do the write.
Certainly... my contention is that if you have a good controller that's
caching writes then drive layout basically won't matter at all, because
the controller will just magically make things optimal.
> From what I've observed in testing, I'd guess that the issue is that
> certain filesystem operations (including, possibly, metadata operations)
> are handled in order. If you have xlog on a seperate partition there
> will never be anything competing with a log write on the server side,
> which won't necessarily be true on a shared filesystem. Even if you have
> a battery backed write cache, you might still have to wait a relatively
> long time for the pg_xlog data to be written out if there's already a
> lot of other stuff in a filesystem write queue.
Well, if the controller is caching with a BBU, I'm not sure that order
matters anymore, because the controller should be able to re-order at
will. Theoretically. :) But this is why having some actual data posted
somewhere would be great.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 16:29 Jim C. Nasby <jnasby@pervasive.com>
parent: Michael Stone <mstone+postgres@mathom.us>
0 siblings, 2 replies; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-15 16:29 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; pgsql-performance
On Mon, Aug 14, 2006 at 01:09:04PM -0400, Michael Stone wrote:
> On Mon, Aug 14, 2006 at 12:05:46PM -0500, Jim C. Nasby wrote:
> >Wow, interesting. IIRC, XFS is lower performing than ext3,
>
> For xlog, maybe. For data, no. Both are definately slower than ext2 for
> xlog, which is another reason to have xlog on a small filesystem which
> doesn't need metadata journalling.
Are 'we' sure that such a setup can't lose any data? I'm worried about
files getting lost when they get written out before the metadata does.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 17:25 Michael Stone <mstone+postgres@mathom.us>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 0 replies; 70+ messages in thread
From: Michael Stone @ 2006-08-15 17:25 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: pgsql-performance
On Tue, Aug 15, 2006 at 11:25:24AM -0500, Jim C. Nasby wrote:
>Well, if the controller is caching with a BBU, I'm not sure that order
>matters anymore, because the controller should be able to re-order at
>will. Theoretically. :) But this is why having some actual data posted
>somewhere would be great.
You're missing the point. It's not a question of what happens once it
gets to the disk/controller, it's a question of whether the xlog write
has to compete with some other write activity before the write gets to
the disk (e.g., at the filesystem level). If you've got a bunch of stuff
in a write buffer on the OS level and you try to push the xlog write
out, you may have to wait for the other stuff to get to the controller
write cache before the xlog does. It doesn't matter if you don't have to
wait for the write to get from the controller cache to the disk if you
already had to wait to get to the controller cache. The effect is a
*lot* smaller than not having a non-volatile cache, but it is an
improvement. (Also, the difference between ext2 and xfs for the xlog is
pretty big itself, and a good reason all by itself to put xlog on a
seperate partition that's small enough to not need journalling.)
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 17:26 Michael Stone <mstone+postgres@mathom.us>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 1 reply; 70+ messages in thread
From: Michael Stone @ 2006-08-15 17:26 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: pgsql-performance
On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote:
>Are 'we' sure that such a setup can't lose any data?
Yes. If you check the archives, you can even find the last time this was
discussed...
The bottom line is that the only reason you need a metadata journalling
filesystem is to save the fsck time when you come up. On a little
partition like xlog, that's not an issue.
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 18:31 mark@mark.mielke.cc
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 1 reply; 70+ messages in thread
From: mark@mark.mielke.cc @ 2006-08-15 18:31 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: Steve Poe <steve.poe@gmail.com>; pgsql-performance
On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote:
> On Mon, Aug 14, 2006 at 01:09:04PM -0400, Michael Stone wrote:
> > On Mon, Aug 14, 2006 at 12:05:46PM -0500, Jim C. Nasby wrote:
> > >Wow, interesting. IIRC, XFS is lower performing than ext3,
> > For xlog, maybe. For data, no. Both are definately slower than ext2 for
> > xlog, which is another reason to have xlog on a small filesystem which
> > doesn't need metadata journalling.
> Are 'we' sure that such a setup can't lose any data? I'm worried about
> files getting lost when they get written out before the metadata does.
I've been worrying about this myself, and my current conclusion is that
ext2 is bad because: a) fsck, and b) data can be lost or corrupted, which
could lead to the need to trash the xlog.
Even ext3 in writeback mode allows for the indirect blocks to be updated
without the data underneath, allowing for blocks to point to random data,
or worse, previous apparently sane data (especially if the data is from
a drive only used for xlog - the chance is high that a block might look
partially valid?).
So, I'm sticking with ext3 in ordered mode.
Cheers,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 18:33 mark@mark.mielke.cc
parent: Michael Stone <mstone+postgres@mathom.us>
0 siblings, 1 reply; 70+ messages in thread
From: mark@mark.mielke.cc @ 2006-08-15 18:33 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; pgsql-performance
On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote:
> On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote:
> >Are 'we' sure that such a setup can't lose any data?
> Yes. If you check the archives, you can even find the last time this was
> discussed...
I looked last night (coincidence actually) and didn't find proof that
you cannot lose data.
How do you deal with the file system structure being updated before the
data blocks are (re-)written?
I don't think you can.
> The bottom line is that the only reason you need a metadata journalling
> filesystem is to save the fsck time when you come up. On a little
> partition like xlog, that's not an issue.
fsck isn't only about time to fix. fsck is needed, because the file system
is broken. If the file system is broken, how can you guarantee data has
not been corrupted?
Cheers,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 19:02 Michael Stone <mstone+postgres@mathom.us>
parent: mark@mark.mielke.cc
0 siblings, 2 replies; 70+ messages in thread
From: Michael Stone @ 2006-08-15 19:02 UTC (permalink / raw)
To: mark@mark.mielke.cc; +Cc: Jim C. Nasby <jnasby@pervasive.com>; pgsql-performance
On Tue, Aug 15, 2006 at 02:33:27PM -0400, mark@mark.mielke.cc wrote:
>On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote:
>> On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote:
>> >Are 'we' sure that such a setup can't lose any data?
>> Yes. If you check the archives, you can even find the last time this was
>> discussed...
>
>I looked last night (coincidence actually) and didn't find proof that
>you cannot lose data.
You aren't going to find proof, any more than you'll find proof that you
won't lose data if you do lose a journalling fs. (Because there isn't
any.) Unfortunately, many people misunderstand the what a metadata
journal does for you, and overstate its importance in this type of
application.
>How do you deal with the file system structure being updated before the
>data blocks are (re-)written?
*That's what the postgres log is for.* If the latest xlog entries don't
make it to disk, they won't be replayed; if they didn't make it to
disk, the transaction would not have been reported as commited. An
application that understands filesystem semantics can guarantee data
integrity without metadata journaling.
>> The bottom line is that the only reason you need a metadata journalling
>> filesystem is to save the fsck time when you come up. On a little
>> partition like xlog, that's not an issue.
>
>fsck isn't only about time to fix. fsck is needed, because the file system
>is broken.
fsck is needed to reconcile the metadata with the on-disk allocations.
To do that, it reads all the inodes and their corresponding directory
entries. The time to do that is proportional to the size of the
filesystem, hence the comment about time. fsck is not needed "because
the filesystem is broken", it's needed because the filesystem is marked
dirty.
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 19:15 Jim C. Nasby <jnasby@pervasive.com>
parent: Michael Stone <mstone+postgres@mathom.us>
1 sibling, 2 replies; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-15 19:15 UTC (permalink / raw)
To: mark@mark.mielke.cc; pgsql-performance
On Tue, Aug 15, 2006 at 03:02:56PM -0400, Michael Stone wrote:
> On Tue, Aug 15, 2006 at 02:33:27PM -0400, mark@mark.mielke.cc wrote:
> >On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote:
> >>On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote:
> >>>Are 'we' sure that such a setup can't lose any data?
> >>Yes. If you check the archives, you can even find the last time this was
> >>discussed...
> >
> >I looked last night (coincidence actually) and didn't find proof that
> >you cannot lose data.
>
> You aren't going to find proof, any more than you'll find proof that you
> won't lose data if you do lose a journalling fs. (Because there isn't
> any.) Unfortunately, many people misunderstand the what a metadata
> journal does for you, and overstate its importance in this type of
> application.
>
> >How do you deal with the file system structure being updated before the
> >data blocks are (re-)written?
>
> *That's what the postgres log is for.* If the latest xlog entries don't
> make it to disk, they won't be replayed; if they didn't make it to
> disk, the transaction would not have been reported as commited. An
> application that understands filesystem semantics can guarantee data
> integrity without metadata journaling.
So what causes files to get 'lost' and get stuck in lost+found?
AFAIK that's because the file was written before the metadata. Now, if
fsync'ing a file also ensures that all the metadata is written, then
we're probably fine... if not, then we're at risk every time we create a
new file (every WAL segment if archiving is on, and every time a
relation passes a 1GB boundary).
FWIW, the way that FreeBSD gets around the need to fsck a dirty
filesystem before use without using a journal is to ensure that metadate
operations are always on the drive before the actual data is written.
There's still a need to fsck a dirty filesystem, but it can now be done
in the background, with the filesystem mounted and in use.
> >>The bottom line is that the only reason you need a metadata journalling
> >>filesystem is to save the fsck time when you come up. On a little
> >>partition like xlog, that's not an issue.
> >
> >fsck isn't only about time to fix. fsck is needed, because the file system
> >is broken.
>
> fsck is needed to reconcile the metadata with the on-disk allocations.
> To do that, it reads all the inodes and their corresponding directory
> entries. The time to do that is proportional to the size of the
> filesystem, hence the comment about time. fsck is not needed "because
> the filesystem is broken", it's needed because the filesystem is marked
> dirty.
>
> Mike Stone
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 19:39 mark@mark.mielke.cc
parent: Michael Stone <mstone+postgres@mathom.us>
1 sibling, 1 reply; 70+ messages in thread
From: mark@mark.mielke.cc @ 2006-08-15 19:39 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; pgsql-performance
On Tue, Aug 15, 2006 at 03:02:56PM -0400, Michael Stone wrote:
> On Tue, Aug 15, 2006 at 02:33:27PM -0400, mark@mark.mielke.cc wrote:
> >>>Are 'we' sure that such a setup can't lose any data?
> >>Yes. If you check the archives, you can even find the last time this was
> >>discussed...
> >I looked last night (coincidence actually) and didn't find proof that
> >you cannot lose data.
> You aren't going to find proof, any more than you'll find proof that you
> won't lose data if you do lose a journalling fs. (Because there isn't
> any.) Unfortunately, many people misunderstand the what a metadata
> journal does for you, and overstate its importance in this type of
> application.
Yes, many people do. :-)
> >How do you deal with the file system structure being updated before the
> >data blocks are (re-)written?
> *That's what the postgres log is for.* If the latest xlog entries don't
> make it to disk, they won't be replayed; if they didn't make it to
> disk, the transaction would not have been reported as commited. An
> application that understands filesystem semantics can guarantee data
> integrity without metadata journaling.
No. This is not true. Updating the file system structure (inodes, indirect
blocks) touches a separate part of the disk than the actual data. If
the file system structure is modified, say, to extend a file to allow
it to contain more data, but the data itself is not written, then upon
a restore, with a system such as ext2, or ext3 with writeback, or xfs,
it is possible that the end of the file, even the postgres log file,
will contain a random block of data from the disk. If this random block
of data happens to look like a valid xlog block, it may be played back,
and the database corrupted.
If the file system is only used for xlog data, the chance that it looks
like a valid block increases, would it not?
> >>The bottom line is that the only reason you need a metadata journalling
> >>filesystem is to save the fsck time when you come up. On a little
> >>partition like xlog, that's not an issue.
> >fsck isn't only about time to fix. fsck is needed, because the file system
> >is broken.
> fsck is needed to reconcile the metadata with the on-disk allocations.
> To do that, it reads all the inodes and their corresponding directory
> entries. The time to do that is proportional to the size of the
> filesystem, hence the comment about time. fsck is not needed "because
> the filesystem is broken", it's needed because the filesystem is marked
> dirty.
This is also wrong. fsck is needed because the file system is broken.
It takes time, because it doesn't have a journal to help it, therefore it
must look through the entire file system and guess what the problems are.
There are classes of problems such as I describe above, for which fsck
*cannot* guess how to solve the problem. There is not enough information
available for it to deduce that anything is wrong at all.
The probability is low, for sure - but then, the chance of a file system
failure is already low.
Betting on ext2 + postgresql xlog has not been confirmed to me as reliable.
Telling me that journalling is misunderstood doesn't prove to me that you
understand it.
I don't mean to be offensive, but I won't accept what you say, as it does
not make sense with my understanding of how file systems work. :-)
Cheers,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 19:42 mark@mark.mielke.cc
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 0 replies; 70+ messages in thread
From: mark@mark.mielke.cc @ 2006-08-15 19:42 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: pgsql-performance
On Tue, Aug 15, 2006 at 02:15:05PM -0500, Jim C. Nasby wrote:
> So what causes files to get 'lost' and get stuck in lost+found?
> AFAIK that's because the file was written before the metadata. Now, if
> fsync'ing a file also ensures that all the metadata is written, then
> we're probably fine... if not, then we're at risk every time we create a
> new file (every WAL segment if archiving is on, and every time a
> relation passes a 1GB boundary).
Only if fsync ensures that the data written to disk is ordered, which as
far as I know, is not done for ext2. Dirty blocks are written in whatever
order is fastest for them to be written, or sequential order, or some
order that isn't based on examining the metadata.
If my understanding is correct - and I've seen nothing yet to say that
it isn't - ext2 is not safe, postgresql xlog or not, fsck or not. It
is safer than no postgresql xlog - but there exists windows, however
small, where the file system can be corrupted.
The need for fsck is due to this problem. If fsck needs to do anything
at all, other than replay a journal, the file system is broken.
Cheers,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 20:05 Tom Lane <tgl@sss.pgh.pa.us>
parent: mark@mark.mielke.cc
0 siblings, 1 reply; 70+ messages in thread
From: Tom Lane @ 2006-08-15 20:05 UTC (permalink / raw)
To: mark@mark.mielke.cc; +Cc: Jim C. Nasby <jnasby@pervasive.com>; Steve Poe <steve.poe@gmail.com>; pgsql-performance
mark@mark.mielke.cc writes:
> I've been worrying about this myself, and my current conclusion is that
> ext2 is bad because: a) fsck, and b) data can be lost or corrupted, which
> could lead to the need to trash the xlog.
> Even ext3 in writeback mode allows for the indirect blocks to be updated
> without the data underneath, allowing for blocks to point to random data,
> or worse, previous apparently sane data (especially if the data is from
> a drive only used for xlog - the chance is high that a block might look
> partially valid?).
At least for xlog, this worrying is misguided, because we zero and fsync
a WAL file before we ever put any valuable data into it. Unless the
filesystem is lying through its teeth about having done an fsync, there
should be no metadata changes happening for an active WAL file (other
than mtime of course).
regards, tom lane
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 20:29 mark@mark.mielke.cc
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 70+ messages in thread
From: mark@mark.mielke.cc @ 2006-08-15 20:29 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Jim C. Nasby <jnasby@pervasive.com>; Steve Poe <steve.poe@gmail.com>; pgsql-performance
On Tue, Aug 15, 2006 at 04:05:17PM -0400, Tom Lane wrote:
> mark@mark.mielke.cc writes:
> > I've been worrying about this myself, and my current conclusion is that
> > ext2 is bad because: a) fsck, and b) data can be lost or corrupted, which
> > could lead to the need to trash the xlog.
> > Even ext3 in writeback mode allows for the indirect blocks to be updated
> > without the data underneath, allowing for blocks to point to random data,
> > or worse, previous apparently sane data (especially if the data is from
> > a drive only used for xlog - the chance is high that a block might look
> > partially valid?).
> At least for xlog, this worrying is misguided, because we zero and fsync
> a WAL file before we ever put any valuable data into it. Unless the
> filesystem is lying through its teeth about having done an fsync, there
> should be no metadata changes happening for an active WAL file (other
> than mtime of course).
Hmmm... I may have missed a post about this in the archive.
WAL file is never appended - only re-written?
If so, then I'm wrong, and ext2 is fine. The requirement is that no
file system structures change as a result of any writes that
PostgreSQL does. If no file system structures change, then I take
everything back as uninformed.
Please confirm whichever. :-)
Cheers,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 20:40 Tom Lane <tgl@sss.pgh.pa.us>
parent: mark@mark.mielke.cc
0 siblings, 0 replies; 70+ messages in thread
From: Tom Lane @ 2006-08-15 20:40 UTC (permalink / raw)
To: mark@mark.mielke.cc; +Cc: Jim C. Nasby <jnasby@pervasive.com>; Steve Poe <steve.poe@gmail.com>; pgsql-performance
mark@mark.mielke.cc writes:
> WAL file is never appended - only re-written?
> If so, then I'm wrong, and ext2 is fine. The requirement is that no
> file system structures change as a result of any writes that
> PostgreSQL does. If no file system structures change, then I take
> everything back as uninformed.
That risk certainly exists in the general data directory, but AFAIK
it's not a problem for pg_xlog.
regards, tom lane
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 20:53 Michael Stone <mstone+postgres@mathom.us>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 0 replies; 70+ messages in thread
From: Michael Stone @ 2006-08-15 20:53 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: mark@mark.mielke.cc; pgsql-performance
On Tue, Aug 15, 2006 at 02:15:05PM -0500, Jim C. Nasby wrote:
>Now, if
>fsync'ing a file also ensures that all the metadata is written, then
>we're probably fine...
...and it does. Unclean shutdowns cause problems in general because
filesystems operate asynchronously. postgres (and other similar
programs) go to great lengths to make sure that critical operations are
performed synchronously. If the program *doesn't* do that, metadata
journaling isn't a magic wand which will guarantee data integrity--it
won't. If the program *does* do that, all the metadata journaling adds
is the ability to skip fsck and start up faster.
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 20:58 Michael Stone <mstone+postgres@mathom.us>
parent: mark@mark.mielke.cc
0 siblings, 1 reply; 70+ messages in thread
From: Michael Stone @ 2006-08-15 20:58 UTC (permalink / raw)
To: mark@mark.mielke.cc; +Cc: pgsql-performance
On Tue, Aug 15, 2006 at 03:39:51PM -0400, mark@mark.mielke.cc wrote:
>No. This is not true. Updating the file system structure (inodes, indirect
>blocks) touches a separate part of the disk than the actual data. If
>the file system structure is modified, say, to extend a file to allow
>it to contain more data, but the data itself is not written, then upon
>a restore, with a system such as ext2, or ext3 with writeback, or xfs,
>it is possible that the end of the file, even the postgres log file,
>will contain a random block of data from the disk. If this random block
>of data happens to look like a valid xlog block, it may be played back,
>and the database corrupted.
you're conflating a whole lot of different issues here. You're ignoring
the fact that postgres preallocates the xlog segment, you're ignoring
the fact that you can sync a directory entry, you're ignoring the fact
that syncing some metadata (such as atime) doesn't matter (only the
block allocation is important in this case, and the blocks are
pre-allocated).
>This is also wrong. fsck is needed because the file system is broken.
nope, the file system *may* be broken. the dirty flag simply indicates
that the filesystem needs to be checked to find out whether or not it is
broken.
>I don't mean to be offensive, but I won't accept what you say, as it does
>not make sense with my understanding of how file systems work. :-)
<shrug> I'm not getting paid to convince you of anything.
Mike Stone
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 21:38 mark@mark.mielke.cc
parent: Michael Stone <mstone+postgres@mathom.us>
0 siblings, 2 replies; 70+ messages in thread
From: mark@mark.mielke.cc @ 2006-08-15 21:38 UTC (permalink / raw)
To: Michael Stone <mstone+postgres@mathom.us>; +Cc: pgsql-performance
On Tue, Aug 15, 2006 at 04:58:59PM -0400, Michael Stone wrote:
> On Tue, Aug 15, 2006 at 03:39:51PM -0400, mark@mark.mielke.cc wrote:
> >No. This is not true. Updating the file system structure (inodes, indirect
> >blocks) touches a separate part of the disk than the actual data. If
> >the file system structure is modified, say, to extend a file to allow
> >it to contain more data, but the data itself is not written, then upon
> >a restore, with a system such as ext2, or ext3 with writeback, or xfs,
> >it is possible that the end of the file, even the postgres log file,
> >will contain a random block of data from the disk. If this random block
> >of data happens to look like a valid xlog block, it may be played back,
> >and the database corrupted.
> you're conflating a whole lot of different issues here. You're ignoring
> the fact that postgres preallocates the xlog segment, you're ignoring
> the fact that you can sync a directory entry, you're ignoring the fact
> that syncing some metadata (such as atime) doesn't matter (only the
> block allocation is important in this case, and the blocks are
> pre-allocated).
Yes, no, no, no. :-)
I didn't know that the xlog segment only uses pre-allocated space. I
ignore mtime/atime as they don't count as file system structure
changes to me. It's updating a field in place. No change to the structure.
With the pre-allocation knowledge, I agree with you. Not sure how I
missed that in my reviewing of the archives... I did know it
pre-allocated once upon a time... Hmm....
> >This is also wrong. fsck is needed because the file system is broken.
> nope, the file system *may* be broken. the dirty flag simply indicates
> that the filesystem needs to be checked to find out whether or not it is
> broken.
Ah, but if we knew it wasn't broken, then fsck wouldn't be needed, now
would it? So we assume that it is broken. A little bit of a game, but
it is important to me. If I assumed the file system was not broken, I
wouldn't run fsck. I run fsck, because I assume it may be broken. If
broken, it indicates potential corruption.
The difference for me, is that if you are correct, that the xlog is
safe, than for a disk that only uses xlog, fsck is not ever necessary,
even after a system crash. If fsck is necessary, then there is potential
for a problem.
With the pre-allocation knowledge, I'm tempted to agree with you that
fsck is not ever necessary for partitions that only hold a properly
pre-allocated xlog.
> >I don't mean to be offensive, but I won't accept what you say, as it does
> >not make sense with my understanding of how file systems work. :-)
> <shrug> I'm not getting paid to convince you of anything.
Just getting you to back up your claim a bit... As I said, no intent
to offend. I learned from it.
Thanks,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 22:20 Jim C. Nasby <jnasby@pervasive.com>
parent: mark@mark.mielke.cc
1 sibling, 1 reply; 70+ messages in thread
From: Jim C. Nasby @ 2006-08-15 22:20 UTC (permalink / raw)
To: mark@mark.mielke.cc; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
On Tue, Aug 15, 2006 at 05:38:43PM -0400, mark@mark.mielke.cc wrote:
> I didn't know that the xlog segment only uses pre-allocated space. I
> ignore mtime/atime as they don't count as file system structure
> changes to me. It's updating a field in place. No change to the structure.
>
> With the pre-allocation knowledge, I agree with you. Not sure how I
> missed that in my reviewing of the archives... I did know it
> pre-allocated once upon a time... Hmm....
This is only valid if the pre-allocation is also fsync'd *and* fsync
ensures that both the metadata and file data are on disk. Anyone
actually checked that? :)
BTW, I did see some anecdotal evidence on one of the lists a while ago.
A PostgreSQL DBA had suggested doing a 'pull the power cord' test to the
other DBAs (all of which were responsible for different RDBMSes,
including a bunch of well known names). They all thought he was off his
rocker. Not too long after that, an unplanned power outage did occur,
and PostgreSQL was the only RDBMS that recovered every single database
without intervention.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-15 22:23 Steinar H. Gunderson <sgunderson@bigfoot.com>
parent: Jim C. Nasby <jnasby@pervasive.com>
0 siblings, 1 reply; 70+ messages in thread
From: Steinar H. Gunderson @ 2006-08-15 22:23 UTC (permalink / raw)
To: pgsql-performance
On Tue, Aug 15, 2006 at 05:20:25PM -0500, Jim C. Nasby wrote:
> This is only valid if the pre-allocation is also fsync'd *and* fsync
> ensures that both the metadata and file data are on disk. Anyone
> actually checked that? :)
fsync() does that, yes. fdatasync() (if it exists), OTOH, doesn't sync the
metadata.
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-16 00:07 David Lang <david@lang.hm>
parent: mark@mark.mielke.cc
1 sibling, 0 replies; 70+ messages in thread
From: David Lang @ 2006-08-16 00:07 UTC (permalink / raw)
To: mark@mark.mielke.cc; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
On Tue, 15 Aug 2006 mark@mark.mielke.cc wrote:
>>> This is also wrong. fsck is needed because the file system is broken.
>> nope, the file system *may* be broken. the dirty flag simply indicates
>> that the filesystem needs to be checked to find out whether or not it is
>> broken.
>
> Ah, but if we knew it wasn't broken, then fsck wouldn't be needed, now
> would it? So we assume that it is broken. A little bit of a game, but
> it is important to me. If I assumed the file system was not broken, I
> wouldn't run fsck. I run fsck, because I assume it may be broken. If
> broken, it indicates potential corruption.
note tha the ext3, reiserfs, jfs, and xfs developers (at least) consider
fsck nessasary even for journaling fileysstems. they just let you get away
without it being mandatory after a unclean shutdown.
David Lang
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-16 03:00 Tom Lane <tgl@sss.pgh.pa.us>
parent: Steinar H. Gunderson <sgunderson@bigfoot.com>
0 siblings, 0 replies; 70+ messages in thread
From: Tom Lane @ 2006-08-16 03:00 UTC (permalink / raw)
To: Steinar H. Gunderson <sgunderson@bigfoot.com>; +Cc: pgsql-performance
"Steinar H. Gunderson" <sgunderson@bigfoot.com> writes:
> On Tue, Aug 15, 2006 at 05:20:25PM -0500, Jim C. Nasby wrote:
>> This is only valid if the pre-allocation is also fsync'd *and* fsync
>> ensures that both the metadata and file data are on disk. Anyone
>> actually checked that? :)
> fsync() does that, yes. fdatasync() (if it exists), OTOH, doesn't sync the
> metadata.
Well, the POSIX spec says that fsync should do that ;-)
My guess is that most/all kernel filesystem layers do indeed try to sync
everything that the spec says they should. The Achilles' heel of the
whole business is disk drives that lie about write completion. The
kernel is just as vulnerable to that as any application ...
regards, tom lane
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-16 08:33 Markus Schaber <schabi@logix-tt.com>
parent: Jim C. Nasby <jnasby@pervasive.com>
1 sibling, 0 replies; 70+ messages in thread
From: Markus Schaber @ 2006-08-16 08:33 UTC (permalink / raw)
To: pgsql-performance
Hi, Jim,
Jim C. Nasby wrote:
> Well, if the controller is caching with a BBU, I'm not sure that order
> matters anymore, because the controller should be able to re-order at
> will. Theoretically. :) But this is why having some actual data posted
> somewhere would be great.
Well, actually, the controller should not reorder over write barriers.
Markus
--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf. | Software Development GIS
Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-16 08:52 Guillaume Cottenceau <gc@mnc.ch>
parent: Luke Lonergan <llonergan@greenplum.com>
3 siblings, 0 replies; 70+ messages in thread
From: Guillaume Cottenceau @ 2006-08-16 08:52 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: pgsql-performance
Hi,
> Can you run bonnie++ version 1.03a on the machine and report the results
> here?
Do you know if the figures from bonnie++ are able to measure the
performance related to the overhead of the 'fsync' option? I had
very strange performance differences between two Dell 1850
machines months ago, and raw performance (hdparm, not bonnie++)
was similar, the only figure I could find with a significant
difference able to explain the issue was the "await" compound
reported by "iostat" - but I was still very much in the dark :/
http://archives.postgresql.org/pgsql-performance/2006-03/msg00407.php
--
Guillaume Cottenceau
Create your personal SMS or WAP Service - visit http://mobilefriends.ch/
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-17 02:10 Steve Poe <steve.poe@gmail.com>
parent: Scott Marlowe <smarlowe@g2switchworks.com>
1 sibling, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-17 02:10 UTC (permalink / raw)
To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: Luke Lonergan <llonergan@greenplum.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Everyone,
I wanted to follow-up on bonnie results for the internal RAID1 which is
connected to the SmartArray 6i. I believe this is the problem, but I am
not good at interepting the results. Here's an sample of three runs:
scsi disc
array ,16G,47983,67,65492,20,37214,6,73785,87,89787,6,578.2,0,16,+++++,
+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
scsi disc
array ,16G,54634,75,67793,21,36835,6,74190,88,89314,6,579.9,0,16,+++++,
+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
scsi disc
array ,16G,55056,76,66108,20,36859,6,74108,87,89559,6,585.0,0,16,+++++,
+++,+++++,+++,+++++,+++,+++++,+++,+
This was run on the internal RAID1 on the outer portion of the discs
formatted at ext2.
Thanks.
Steve
On Thu, 2006-08-10 at 10:35 -0500, Scott Marlowe wrote:
> On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote:
> > Mike,
> >
> > On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
> >
> > > On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
> > >> I tried as you suggested and my performance dropped by 50%. I went from
> > >> a 32 TPS to 16. Oh well.
> > >
> > > If you put data & xlog on the same array, put them on seperate
> > > partitions, probably formatted differently (ext2 on xlog).
> >
> > If he's doing the same thing on both systems (Sun and HP) and the HP
> > performance is dramatically worse despite using more disks and having faster
> > CPUs and more RAM, ISTM the problem isn't the configuration.
> >
> > Add to this the fact that the Sun machine is CPU bound while the HP is I/O
> > wait bound and I think the problem is the disk hardware or the driver
> > therein.
>
> I agree. The problem here looks to be the RAID controller.
>
> Steve, got access to a different RAID controller to test with?
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 10:50 Magnus Hagander <mha@sollentuna.net>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 0 replies; 70+ messages in thread
From: Magnus Hagander @ 2006-08-18 10:50 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; Joshua D. Drake <jd@commandprompt.com>; +Cc: Luke Lonergan <llonergan@greenplum.com>; pgsql-performance
> There is 64MB on the 6i and 192MB on the 642 controller. I wish the
> controllers had a "wrieback" enable option like the LSI MegaRAID
> adapters have. I have tried splitting the cache accelerator 25/75
> 75/25 0/100 100/0 but the results really did not improve.
They have a writeback option, but you can't enable it unless you buy the
battery-pack for the controller. I believe it's enabled by default once
you get the BBWC.
//Magnus
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 14:37 Luke Lonergan <llonergan@greenplum.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 2 replies; 70+ messages in thread
From: Luke Lonergan @ 2006-08-18 14:37 UTC (permalink / raw)
To: steve.poe@gmail.com, "Scott Marlowe" <smarlowe@g2switchworks.com>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Steve,
If this is an internal RAID1 on two disks, it looks great.
Based on the random seeks though (578 seeks/sec), it looks like maybe it's 6
disks in a RAID10?
- Luke
On 8/16/06 7:10 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
> Everyone,
>
> I wanted to follow-up on bonnie results for the internal RAID1 which is
> connected to the SmartArray 6i. I believe this is the problem, but I am
> not good at interepting the results. Here's an sample of three runs:
>
> scsi disc
> array ,16G,47983,67,65492,20,37214,6,73785,87,89787,6,578.2,0,16,+++++,
> +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
> scsi disc
> array ,16G,54634,75,67793,21,36835,6,74190,88,89314,6,579.9,0,16,+++++,
> +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
> scsi disc
> array ,16G,55056,76,66108,20,36859,6,74108,87,89559,6,585.0,0,16,+++++,
> +++,+++++,+++,+++++,+++,+++++,+++,+
>
> This was run on the internal RAID1 on the outer portion of the discs
> formatted at ext2.
>
> Thanks.
>
> Steve
>
> On Thu, 2006-08-10 at 10:35 -0500, Scott Marlowe wrote:
>> On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote:
>>> Mike,
>>>
>>> On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
>>>
>>>> On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
>>>>> I tried as you suggested and my performance dropped by 50%. I went from
>>>>> a 32 TPS to 16. Oh well.
>>>>
>>>> If you put data & xlog on the same array, put them on seperate
>>>> partitions, probably formatted differently (ext2 on xlog).
>>>
>>> If he's doing the same thing on both systems (Sun and HP) and the HP
>>> performance is dramatically worse despite using more disks and having faster
>>> CPUs and more RAM, ISTM the problem isn't the configuration.
>>>
>>> Add to this the fact that the Sun machine is CPU bound while the HP is I/O
>>> wait bound and I think the problem is the disk hardware or the driver
>>> therein.
>>
>> I agree. The problem here looks to be the RAID controller.
>>
>> Steve, got access to a different RAID controller to test with?
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 1: if posting/reading through Usenet, please send an appropriate
>> subscribe-nomail command to majordomo@postgresql.org so that your
>> message can get through to the mailing list cleanly
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 15:26 Bucky Jordan <bjordan@lumeta.com>
parent: Luke Lonergan <llonergan@greenplum.com>
1 sibling, 0 replies; 70+ messages in thread
From: Bucky Jordan @ 2006-08-18 15:26 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: pgsql-performance
That's about what I was getting for a 2 disk RAID 0 setup on a PE 2950.
Here's bonnie++ numbers for the RAID10x4 and RAID0x2, unfortunately I
only have the 1.93 numbers since this was before I got the advice to run
with the earlier version of bonnie and larger file sizes, so I don't
know how meaningful they are.
RAID 10x4
bash-2.05b$ bonnie++ -d bonnie -s 1000:8k
Version 1.93c ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
1000M 585 99 21705 4 28560 9 1004 99 812997 98 5436
454
Latency 14181us 81364us 50256us 57720us 1671us
1059ms
Version 1.93c ------Sequential Create------ --------Random
Create--------
c -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
/sec %CP
16 4712 10 +++++ +++ +++++ +++ 4674 10 +++++ +++
+++++ +++
Latency 807ms 21us 36us 804ms 110us
36us
1.93c,1.93c,
,1,1155207445,1000M,,585,99,21705,4,28560,9,1004,99,812997,98,5436,454,1
6,,,,,4712,10,+++++,+++,+++++,+++,4674,10,+++++,+++,+++++,+++,14181us,81
364us,50256us,57720us,1671us,1059ms,807ms,21us,36us,804ms,110us,36us
bash-2.05b$
RAID 0x2
bash-2.05b$ bonnie++ -d bonnie -s 1000:8k
Version 1.93c ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
1000M 575 99 131621 25 104178 26 1004 99 816928 99 6233
521
Latency 14436us 26663us 47478us 54796us 1487us
38924us
Version 1.93c ------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
/sec %CP
16 4935 10 +++++ +++ +++++ +++ 5198 11 +++++ +++
+++++ +++
Latency 738ms 32us 43us 777ms 24us
30us
1.93c,1.93c,beast.corp.lumeta.com,1,1155210203,1000M,,575,99,131621,25,1
04178,26,1004,99,816928,99,6233,521,16,,,,,4935,10,+++++,+++,+++++,+++,5
198,11,+++++,+++,+++++,+++,14436us,26663us,47478us,54796us,1487us,38924u
s,738ms,32us,43us,777ms,24us,30us
A RAID 5 configuration seems to outperform this on the PE 2950 though
(at least in terms of raw read/write perf)
If anyone's interested in some more detailed tests of the 2950, I might
be able to reconfigure the raid for some tests next week before I start
setting up the box for long term use, so I'm open to suggestions. See
earlier posts in this thread for details about the hardware.
Thanks,
Bucky
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Luke
Lonergan
Sent: Friday, August 18, 2006 10:38 AM
To: steve.poe@gmail.com; Scott Marlowe
Cc: Michael Stone; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Postgresql Performance on an HP DL385 and
Steve,
If this is an internal RAID1 on two disks, it looks great.
Based on the random seeks though (578 seeks/sec), it looks like maybe
it's 6
disks in a RAID10?
- Luke
On 8/16/06 7:10 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
> Everyone,
>
> I wanted to follow-up on bonnie results for the internal RAID1 which
is
> connected to the SmartArray 6i. I believe this is the problem, but I
am
> not good at interepting the results. Here's an sample of three runs:
>
> scsi disc
> array
,16G,47983,67,65492,20,37214,6,73785,87,89787,6,578.2,0,16,+++++,
> +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
> scsi disc
> array
,16G,54634,75,67793,21,36835,6,74190,88,89314,6,579.9,0,16,+++++,
> +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
> scsi disc
> array
,16G,55056,76,66108,20,36859,6,74108,87,89559,6,585.0,0,16,+++++,
> +++,+++++,+++,+++++,+++,+++++,+++,+
>
> This was run on the internal RAID1 on the outer portion of the discs
> formatted at ext2.
>
> Thanks.
>
> Steve
>
> On Thu, 2006-08-10 at 10:35 -0500, Scott Marlowe wrote:
>> On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote:
>>> Mike,
>>>
>>> On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us>
wrote:
>>>
>>>> On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
>>>>> I tried as you suggested and my performance dropped by 50%. I went
from
>>>>> a 32 TPS to 16. Oh well.
>>>>
>>>> If you put data & xlog on the same array, put them on seperate
>>>> partitions, probably formatted differently (ext2 on xlog).
>>>
>>> If he's doing the same thing on both systems (Sun and HP) and the HP
>>> performance is dramatically worse despite using more disks and
having faster
>>> CPUs and more RAM, ISTM the problem isn't the configuration.
>>>
>>> Add to this the fact that the Sun machine is CPU bound while the HP
is I/O
>>> wait bound and I think the problem is the disk hardware or the
driver
>>> therein.
>>
>> I agree. The problem here looks to be the RAID controller.
>>
>> Steve, got access to a different RAID controller to test with?
>>
>> ---------------------------(end of
broadcast)---------------------------
>> TIP 1: if posting/reading through Usenet, please send an appropriate
>> subscribe-nomail command to majordomo@postgresql.org so that
your
>> message can get through to the mailing list cleanly
>
>
---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 17:39 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
1 sibling, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-18 17:39 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Luke,
Nope. it is only a RAID1 for the 2 internal discs connected to the
SmartArray 6i. This is where I *had* the pg_xlog located when the
performance was very poor. Also, I just found out the default stripe size is
128k. Would this be a problem for pg_xlog?
The 6-disc RAID10 you speak of is on the SmartArray 642 RAID adapter.
Steve
On 8/18/06, Luke Lonergan <llonergan@greenplum.com> wrote:
>
> Steve,
>
> If this is an internal RAID1 on two disks, it looks great.
>
> Based on the random seeks though (578 seeks/sec), it looks like maybe it's
> 6
> disks in a RAID10?
>
> - Luke
>
>
> On 8/16/06 7:10 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
>
> > Everyone,
> >
> > I wanted to follow-up on bonnie results for the internal RAID1 which is
> > connected to the SmartArray 6i. I believe this is the problem, but I am
> > not good at interepting the results. Here's an sample of three runs:
> >
> > scsi disc
> > array ,16G,47983,67,65492,20,37214,6,73785,87,89787,6,578.2,0,16,+++++,
> > +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
> > scsi disc
> > array ,16G,54634,75,67793,21,36835,6,74190,88,89314,6,579.9,0,16,+++++,
> > +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
> > scsi disc
> > array ,16G,55056,76,66108,20,36859,6,74108,87,89559,6,585.0,0,16,+++++,
> > +++,+++++,+++,+++++,+++,+++++,+++,+
> >
> > This was run on the internal RAID1 on the outer portion of the discs
> > formatted at ext2.
> >
> > Thanks.
> >
> > Steve
> >
> > On Thu, 2006-08-10 at 10:35 -0500, Scott Marlowe wrote:
> >> On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote:
> >>> Mike,
> >>>
> >>> On 8/10/06 4:09 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
> >>>
> >>>> On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
> >>>>> I tried as you suggested and my performance dropped by 50%. I went
> from
> >>>>> a 32 TPS to 16. Oh well.
> >>>>
> >>>> If you put data & xlog on the same array, put them on seperate
> >>>> partitions, probably formatted differently (ext2 on xlog).
> >>>
> >>> If he's doing the same thing on both systems (Sun and HP) and the HP
> >>> performance is dramatically worse despite using more disks and having
> faster
> >>> CPUs and more RAM, ISTM the problem isn't the configuration.
> >>>
> >>> Add to this the fact that the Sun machine is CPU bound while the HP is
> I/O
> >>> wait bound and I think the problem is the disk hardware or the driver
> >>> therein.
> >>
> >> I agree. The problem here looks to be the RAID controller.
> >>
> >> Steve, got access to a different RAID controller to test with?
> >>
> >> ---------------------------(end of
> broadcast)---------------------------
> >> TIP 1: if posting/reading through Usenet, please send an appropriate
> >> subscribe-nomail command to majordomo@postgresql.org so that
> your
> >> message can get through to the mailing list cleanly
> >
> >
>
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 18:09 Luke Lonergan <llonergan@greenplum.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-18 18:09 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Steve,
On 8/18/06 10:39 AM, "Steve Poe" <steve.poe@gmail.com> wrote:
> Nope. it is only a RAID1 for the 2 internal discs connected to the SmartArray
> 6i. This is where I *had* the pg_xlog located when the performance was very
> poor. Also, I just found out the default stripe size is 128k. Would this be a
> problem for pg_xlog?
ISTM that the main performance issue for xlog is going to be the rate at
which fdatasync operations complete, and the stripe size shouldn't hurt
that.
What are your postgresql.conf settings for the xlog: how many logfiles,
sync_method, etc?
> The 6-disc RAID10 you speak of is on the SmartArray 642 RAID adapter.
Interesting - the seek rate is very good for two drives, are they 15K RPM?
- Luke
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 19:00 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
0 siblings, 1 reply; 70+ messages in thread
From: Steve Poe @ 2006-08-18 19:00 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Luke,
ISTM that the main performance issue for xlog is going to be the rate at
> which fdatasync operations complete, and the stripe size shouldn't hurt
> that.
I thought so. However, I've also tried running the PGDATA off of the RAID1
as a test and it is poor.
What are your postgresql.conf settings for the xlog: how many logfiles,
> sync_method, etc?
wal_sync_method = fsync # the default varies across platforms:
# fsync, fdatasync, open_sync, or
open_datasync
# - Checkpoints -
checkpoint_segments = 14 # in logfile segments, min 1, 16MB each
checkpoint_timeout = 300 # range 30-3600, in seconds
#checkpoint_warning = 30 # 0 is off, in seconds
#commit_delay = 0 # range 0-100000, in microseconds
#commit_siblings = 5
What stumps me is I use the same settings on a Sun box (dual Opteron 4GB w/
LSI MegaRAID 128M) with the same data. This is on pg 7.4.13.
> The 6-disc RAID10 you speak of is on the SmartArray 642 RAID adapter.
>
> Interesting - the seek rate is very good for two drives, are they 15K RPM?
Nope. 10K. RPM.
HP's recommendation for testing is to connect the RAID1 to the second
channel off of the SmartArray 642 adapter since they use the same driver,
and, according to HP, I should not have to rebuilt the RAID1.
I have to send the new server to the hospital next week, so I have very
little testing time left.
Steve
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 21:32 Luke Lonergan <llonergan@greenplum.com>
parent: Steve Poe <steve.poe@gmail.com>
0 siblings, 1 reply; 70+ messages in thread
From: Luke Lonergan @ 2006-08-18 21:32 UTC (permalink / raw)
To: Steve Poe <steve.poe@gmail.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Steve,
One thing here is that ³wal_sync_method² should be set to ³fdatasync² and
not ³fsync². In fact, the default is fdatasync, but because you have
uncommented the standard line in the file, it is changed to ³fsync², which
is a lot slower. This is a bug in the file defaults.
That could speed things up quite a bit on the xlog.
WRT the difference between the two systems, I¹m kind of stumped.
- Luke
On 8/18/06 12:00 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
>
> Luke,
>
>> ISTM that the main performance issue for xlog is going to be the rate at
>> which fdatasync operations complete, and the stripe size shouldn't hurt
>> that.
>
> I thought so. However, I've also tried running the PGDATA off of the RAID1 as
> a test and it is poor.
>
>
>
>> What are your postgresql.conf settings for the xlog: how many logfiles,
>> sync_method, etc?
>
> wal_sync_method = fsync # the default varies across platforms:
> # fsync, fdatasync, open_sync, or
> open_datasync
> # - Checkpoints -
>
> checkpoint_segments = 14 # in logfile segments, min 1, 16MB each
> checkpoint_timeout = 300 # range 30-3600, in seconds
> #checkpoint_warning = 30 # 0 is off, in seconds
> #commit_delay = 0 # range 0-100000, in microseconds
> #commit_siblings = 5
>
> What stumps me is I use the same settings on a Sun box (dual Opteron 4GB w/
> LSI MegaRAID 128M) with the same data. This is on pg 7.4.13.
>
>
>
>>> > The 6-disc RAID10 you speak of is on the SmartArray 642 RAID adapter.
>>
>> Interesting - the seek rate is very good for two drives, are they 15K RPM?
>
> Nope. 10K. RPM.
>
>
> HP's recommendation for testing is to connect the RAID1 to the second channel
> off of the SmartArray 642 adapter since they use the same driver, and,
> according to HP, I should not have to rebuilt the RAID1.
>
> I have to send the new server to the hospital next week, so I have very little
> testing time left.
>
> Steve
>
>
>
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Postgresql Performance on an HP DL385 and
@ 2006-08-18 22:23 Steve Poe <steve.poe@gmail.com>
parent: Luke Lonergan <llonergan@greenplum.com>
0 siblings, 0 replies; 70+ messages in thread
From: Steve Poe @ 2006-08-18 22:23 UTC (permalink / raw)
To: Luke Lonergan <llonergan@greenplum.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Luke,
I'll try it, but you're right, it should not matter. The two systems are:
HP DL385 (dual Opteron 265 I believe) 8GB of RAM, two internal RAID1 U320
10K
Sun W2100z (dual Opteron 245 I believe) 4GB of RAM, 1 U320 10K drive with
LSI MegaRAID 2X 128M driving two external 4-disc arrays U320 10K drives in a
RAID10 configuration. Running same version of LInux (Centos 4.3 ) and same
kernel version. No changes within the kernel for each of them. Running the
same *.conf files for Postgresql 7.4.13.
Steve
On 8/18/06, Luke Lonergan <llonergan@greenplum.com> wrote:
>
> Steve,
>
> One thing here is that "wal_sync_method" should be set to "fdatasync" and
> not "fsync". In fact, the default is fdatasync, but because you have
> uncommented the standard line in the file, it is changed to "fsync", which
> is a lot slower. This is a bug in the file defaults.
>
> That could speed things up quite a bit on the xlog.
>
> WRT the difference between the two systems, I'm kind of stumped.
>
> - Luke
>
>
> On 8/18/06 12:00 PM, "Steve Poe" <steve.poe@gmail.com> wrote:
>
>
> Luke,
>
> ISTM that the main performance issue for xlog is going to be the rate at
> which fdatasync operations complete, and the stripe size shouldn't hurt
> that.
>
>
> I thought so. However, I've also tried running the PGDATA off of the RAID1
> as a test and it is poor.
>
>
>
> What are your postgresql.conf settings for the xlog: how many logfiles,
> sync_method, etc?
>
>
> wal_sync_method = fsync # the default varies across platforms:
> # fsync, fdatasync, open_sync, or
> open_datasync
> # - Checkpoints -
>
> checkpoint_segments = 14 # in logfile segments, min 1, 16MB each
> checkpoint_timeout = 300 # range 30-3600, in seconds
> #checkpoint_warning = 30 # 0 is off, in seconds
> #commit_delay = 0 # range 0-100000, in microseconds
> #commit_siblings = 5
>
> What stumps me is I use the same settings on a Sun box (dual Opteron 4GB
> w/ LSI MegaRAID 128M) with the same data. This is on pg 7.4.13.
>
>
>
> > The 6-disc RAID10 you speak of is on the SmartArray 642 RAID adapter.
>
> Interesting - the seek rate is very good for two drives, are they 15K RPM?
>
>
> Nope. 10K. RPM.
>
>
> HP's recommendation for testing is to connect the RAID1 to the second
> channel off of the SmartArray 642 adapter since they use the same driver,
> and, according to HP, I should not have to rebuilt the RAID1.
>
> I have to send the new server to the hospital next week, so I have very
> little testing time left.
>
> Steve
>
>
>
>
>
>
>
^ permalink raw reply [nested|flat] 70+ messages in thread
end of thread, other threads:[~2006-08-18 22:23 UTC | newest]
Thread overview: 70+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-08-05 23:10 Postgresql Performance on an HP DL385 and SmartArray 642 Steve Poe <steve.poe@gmail.com>
2006-08-08 01:10 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-08 01:46 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-08 02:19 ` Re: Postgresql Performance on an HP DL385 and Joshua D. Drake <jd@commandprompt.com>
2006-08-08 02:28 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-18 10:50 ` Re: Postgresql Performance on an HP DL385 and Magnus Hagander <mha@sollentuna.net>
2006-08-08 04:13 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-08 06:40 ` Re: Postgresql Performance on an HP DL385 and Alex Turner <armtuk@gmail.com>
2006-08-08 13:57 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-16 08:52 ` Re: Postgresql Performance on an HP DL385 and Guillaume Cottenceau <gc@mnc.ch>
2006-08-08 12:20 ` Jeff Trout <threshar@torgo.978.org>
2006-08-08 14:36 Re: Postgresql Performance on an HP DL385 and Luke Lonergan <LLonergan@greenplum.com>
2006-08-08 15:01 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-08 16:22 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-08 16:57 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 01:55 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-09 04:56 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 05:22 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 05:22 Re: Postgresql Performance on an HP DL385 and Luke Lonergan <LLonergan@greenplum.com>
2006-08-09 05:45 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 06:33 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 21:05 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-09 21:11 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 21:37 ` Re: Postgresql Performance on an HP DL385 and Scott Marlowe <smarlowe@g2switchworks.com>
2006-08-09 23:47 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-10 01:24 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-10 03:29 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-10 11:09 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-10 15:15 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-10 15:35 ` Re: Postgresql Performance on an HP DL385 and Scott Marlowe <smarlowe@g2switchworks.com>
2006-08-10 15:47 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-17 02:10 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-18 14:37 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-18 15:26 ` Re: Postgresql Performance on an HP DL385 and Bucky Jordan <bjordan@lumeta.com>
2006-08-18 17:39 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-18 18:09 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-18 19:00 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-18 21:32 ` Re: Postgresql Performance on an HP DL385 and Luke Lonergan <llonergan@greenplum.com>
2006-08-18 22:23 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-14 15:38 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-14 15:51 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-14 17:05 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-14 17:09 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-15 16:29 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-15 17:26 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-15 18:33 ` Re: Postgresql Performance on an HP DL385 and mark@mark.mielke.cc
2006-08-15 19:02 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-15 19:15 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-15 19:42 ` Re: Postgresql Performance on an HP DL385 and mark@mark.mielke.cc
2006-08-15 20:53 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-15 19:39 ` Re: Postgresql Performance on an HP DL385 and mark@mark.mielke.cc
2006-08-15 20:58 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-15 21:38 ` Re: Postgresql Performance on an HP DL385 and mark@mark.mielke.cc
2006-08-15 22:20 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-15 22:23 ` Re: Postgresql Performance on an HP DL385 and Steinar H. Gunderson <sgunderson@bigfoot.com>
2006-08-16 03:00 ` Re: Postgresql Performance on an HP DL385 and Tom Lane <tgl@sss.pgh.pa.us>
2006-08-16 00:07 ` Re: Postgresql Performance on an HP DL385 and David Lang <david@lang.hm>
2006-08-15 18:31 ` Re: Postgresql Performance on an HP DL385 and mark@mark.mielke.cc
2006-08-15 20:05 ` Re: Postgresql Performance on an HP DL385 and Tom Lane <tgl@sss.pgh.pa.us>
2006-08-15 20:29 ` Re: Postgresql Performance on an HP DL385 and mark@mark.mielke.cc
2006-08-15 20:40 ` Re: Postgresql Performance on an HP DL385 and Tom Lane <tgl@sss.pgh.pa.us>
2006-08-14 17:03 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-15 16:25 ` Re: Postgresql Performance on an HP DL385 and Jim C. Nasby <jnasby@pervasive.com>
2006-08-15 17:25 ` Re: Postgresql Performance on an HP DL385 and Michael Stone <mstone+postgres@mathom.us>
2006-08-16 08:33 ` Re: Postgresql Performance on an HP DL385 and Markus Schaber <schabi@logix-tt.com>
2006-08-09 05:24 Re: Postgresql Performance on an HP DL385 and Luke Lonergan <LLonergan@greenplum.com>
2006-08-09 06:35 Re: Postgresql Performance on an HP DL385 and Luke Lonergan <LLonergan@greenplum.com>
2006-08-09 06:47 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
2006-08-09 06:50 Re: Postgresql Performance on an HP DL385 and Luke Lonergan <LLonergan@greenplum.com>
2006-08-09 06:56 ` Re: Postgresql Performance on an HP DL385 and Steve Poe <steve.poe@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox