pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedbest arrangement of 3 disks for (insert) performance
14+ messages / 10 participants
[nested] [flat]
* best arrangement of 3 disks for (insert) performance
@ 2003-09-12 10:26 Richard Jones <rj@last.fm>
2003-09-12 11:06 ` Re: best arrangement of 3 disks for (insert) performance Matt Clark <matt@ymogen.net>
2003-09-12 15:24 ` Re: best arrangement of 3 disks for (insert) performance Christopher Browne <cbbrowne@libertyrms.info>
2003-09-12 16:49 ` Re: best arrangement of 3 disks for (insert) performance Josh Berkus <josh@agliodbs.com>
0 siblings, 3 replies; 14+ messages in thread
From: Richard Jones @ 2003-09-12 10:26 UTC (permalink / raw)
To: pgsql-performance
Hi all,
I have some new hardware on the way and would like some advice on how to get
the most out of it..
its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks
should i mirror 2 of the disks for postgres data, and use the 3rd disk for the
o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg and
use the 3rd for o/s,logs,backups ?
the machine will be dealing with lots of inserts, basically as many as we can
throw at it
thanks,
Richard
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
@ 2003-09-12 11:06 ` Matt Clark <matt@ymogen.net>
2 siblings, 0 replies; 14+ messages in thread
From: Matt Clark @ 2003-09-12 11:06 UTC (permalink / raw)
To: rj@last.fm; pgsql-performance
> the machine will be dealing with lots of inserts, basically as many as we can
> throw at it
If you mean lots of _transactions_ with few inserts per transaction you should get a RAID controller w/ battery backed write-back
cache. Nothing else will improve your write performance by nearly as much. You could sell the RAM and one of the CPU's to pay for
it ;-)
If you have lots of inserts but all in a few transactions then it's not quite so critical.
M
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
@ 2003-09-12 15:24 ` Christopher Browne <cbbrowne@libertyrms.info>
2003-09-12 16:55 ` Re: best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-14 04:44 ` Re: best arrangement of 3 disks for (insert) performance Cott Lang <cott@internetstaff.com>
2 siblings, 2 replies; 14+ messages in thread
From: Christopher Browne @ 2003-09-12 15:24 UTC (permalink / raw)
To: pgsql-performance
rj@last.fm (Richard Jones) writes:
> I have some new hardware on the way and would like some advice on
> how to get the most out of it..
>
> its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks
>
> should i mirror 2 of the disks for postgres data, and use the 3rd
> disk for the o/s and the pg logs or raid5 the 3 disks or even stripe
> 2 disks for pg and use the 3rd for o/s,logs,backups ?
>
> the machine will be dealing with lots of inserts, basically as many
> as we can throw at it
Having WAL on a separate drive from the database would be something of
a win. I'd buy that 1 disk for OS+WAL and then RAID [something]
across the other two drives for the database would be pretty helpful.
After doing some [loose] benchmarking, the VERY best way to improve
performance would involve a RAID controller with battery-backed cache.
On a box with similar configuration to yours, it took ~3h for a
particular set of data to load; on another one with battery-backed
cache (and a dozen fast SCSI drives :-)), the same data took as little
as 6 minutes to load. The BIG effect seemed to come from the
controller.
--
(reverse (concatenate 'string "ofni.smrytrebil" "@" "enworbbc"))
<http://dev6.int.libertyrms.com/;
Christopher Browne
(416) 646 3304 x124 (land)
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-12 15:24 ` Re: best arrangement of 3 disks for (insert) performance Christopher Browne <cbbrowne@libertyrms.info>
@ 2003-09-12 16:55 ` Richard Jones <rj@last.fm>
2003-09-12 22:11 ` Re: best arrangement of 3 disks for (insert) performance Rod Taylor <rbt@rbt.ca>
1 sibling, 1 reply; 14+ messages in thread
From: Richard Jones @ 2003-09-12 16:55 UTC (permalink / raw)
To: Christopher Browne <cbbrowne@libertyrms.info>; pgsql-performance
The machine is coming from dell, and i have the option of a
PERC 3/SC RAID Controller (32MB)
or software raid.
does anyone have any experience of this controller?
its an additional £345 for this controller, i'd be interested to know what
people think - my other option is to buy the raid controller separately,
which appeals to me but i wouldnt know what to look for in a raid controller.
that raid controller review site sounds like a good idea :)
Richard.
On Friday 12 September 2003 4:24 pm, Christopher Browne wrote:
> rj@last.fm (Richard Jones) writes:
> > I have some new hardware on the way and would like some advice on
> > how to get the most out of it..
> >
> > its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks
> >
> > should i mirror 2 of the disks for postgres data, and use the 3rd
> > disk for the o/s and the pg logs or raid5 the 3 disks or even stripe
> > 2 disks for pg and use the 3rd for o/s,logs,backups ?
> >
> > the machine will be dealing with lots of inserts, basically as many
> > as we can throw at it
>
> Having WAL on a separate drive from the database would be something of
> a win. I'd buy that 1 disk for OS+WAL and then RAID [something]
> across the other two drives for the database would be pretty helpful.
>
> After doing some [loose] benchmarking, the VERY best way to improve
> performance would involve a RAID controller with battery-backed cache.
>
> On a box with similar configuration to yours, it took ~3h for a
> particular set of data to load; on another one with battery-backed
> cache (and a dozen fast SCSI drives :-)), the same data took as little
> as 6 minutes to load. The BIG effect seemed to come from the
> controller.
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-12 15:24 ` Re: best arrangement of 3 disks for (insert) performance Christopher Browne <cbbrowne@libertyrms.info>
2003-09-12 16:55 ` Re: best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
@ 2003-09-12 22:11 ` Rod Taylor <rbt@rbt.ca>
0 siblings, 0 replies; 14+ messages in thread
From: Rod Taylor @ 2003-09-12 22:11 UTC (permalink / raw)
To: rj@last.fm; +Cc: Christopher Browne <cbbrowne@libertyrms.info>; pgsql-performance
On Fri, 2003-09-12 at 12:55, Richard Jones wrote:
> The machine is coming from dell, and i have the option of a
> PERC 3/SC RAID Controller (32MB)
> or software raid.
>
> does anyone have any experience of this controller?
> its an additional £345 for this controller, i'd be interested to know what
> people think - my other option is to buy the raid controller separately,
> which appeals to me but i wouldnt know what to look for in a raid controller.
Hardware raid with the write cache, and sell a CPU if necessary to buy
it (don't sell the ram though!).
Attachments:
signature.asc (application/pgp-signature, 186B) download
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-12 15:24 ` Re: best arrangement of 3 disks for (insert) performance Christopher Browne <cbbrowne@libertyrms.info>
@ 2003-09-14 04:44 ` Cott Lang <cott@internetstaff.com>
2003-09-15 08:05 ` Re: [PERFORM] best arrangement of 3 disks for (insert) performance Ron Johnson <ron.l.johnson@cox.net>
1 sibling, 1 reply; 14+ messages in thread
From: Cott Lang @ 2003-09-14 04:44 UTC (permalink / raw)
To: pgsql-performance
> Having WAL on a separate drive from the database would be something of
> a win. I'd buy that 1 disk for OS+WAL and then RAID [something]
> across the other two drives for the database would be pretty helpful.
Just my .02,
I did a lot of testing before I deployed our ~50GB postgresql databases
with various combinations of 6 15k SCSI drives. I did custom benchmarks
to simulate our applications, I downloaded several benchmarks, etc.
It might be a fluke, but I never got better performance with WALs on a
different disk than I did with all 6 disks in a 0+1 configuration.
Obviously that's not an option with 3 disks. =)
I ended up going with that for easier space maintenance.
Obviously YMMV, benchmark for your own situation. :)
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: [PERFORM] best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-12 15:24 ` Re: best arrangement of 3 disks for (insert) performance Christopher Browne <cbbrowne@libertyrms.info>
2003-09-14 04:44 ` Re: best arrangement of 3 disks for (insert) performance Cott Lang <cott@internetstaff.com>
@ 2003-09-15 08:05 ` Ron Johnson <ron.l.johnson@cox.net>
0 siblings, 0 replies; 14+ messages in thread
From: Ron Johnson @ 2003-09-15 08:05 UTC (permalink / raw)
To: PgSQL General ML <pgsql-general@postgresql.org>
On Sat, 2003-09-13 at 23:44, Cott Lang wrote:
> > Having WAL on a separate drive from the database would be something of
> > a win. I'd buy that 1 disk for OS+WAL and then RAID [something]
> > across the other two drives for the database would be pretty helpful.
>
> Just my .02,
>
> I did a lot of testing before I deployed our ~50GB postgresql databases
> with various combinations of 6 15k SCSI drives. I did custom benchmarks
> to simulate our applications, I downloaded several benchmarks, etc.
>
> It might be a fluke, but I never got better performance with WALs on a
> different disk than I did with all 6 disks in a 0+1 configuration.
> Obviously that's not an option with 3 disks. =)
Interesting. Where did you put the OS, and what kind of, and how
many, SCSI controllers did you have?
PCI 32bit/33MHz or 64bit/66MHz PCI? 32bit/33MHz PCI has a max
throughput of 132MB/s, which is 60% smaller than the theoretical
bandwidth of U320 SCSI, so maybe you were saturating the PCI bus,
and that's why a separate WAL didn't show any improvement?
(If the WAL ever becomes the vehicle for PITR, then it will have
to be on a separate disk [and preferably a separate controller],
even if it slows performance.)
--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@cox.net
Jefferson, LA USA
"Eternal vigilance is the price of liberty: power is ever
stealing from the many to the few. The manna of popular liberty
must be gathered each day, or it is rotten... The hand entrusted
with power becomes, either from human depravity or esprit de
corps, the necessary enemy of the people. Only by continual
oversight can the democrat in office be prevented from hardening
into a despot: only by unintermitted agitation can a people be
kept sufficiently awake to principle not to let liberty be
smothered in material prosperity... Never look, for an age when
the people can be quiet and safe. At such times despotism, like
a shrouding mist, steals over the mirror of Freedom"
Wendell Phillips
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
@ 2003-09-12 16:49 ` Josh Berkus <josh@agliodbs.com>
2003-09-12 16:57 ` Re: best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2 siblings, 1 reply; 14+ messages in thread
From: Josh Berkus @ 2003-09-12 16:49 UTC (permalink / raw)
To: rj@last.fm; pgsql-performance
RIchard,
> its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks
>
> should i mirror 2 of the disks for postgres data, and use the 3rd disk for
the
> o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg and
> use the 3rd for o/s,logs,backups ?
I'd mirror 2. Stripey RAID with few disks imposes a heavy performance
penalty on data writes (particularly updates), sometimes as much as 50% for a
RAID5-3disk config.
I am a little curious why you've got a dual-xeon, but could only afford 3
disks ....
--
-Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-12 16:49 ` Re: best arrangement of 3 disks for (insert) performance Josh Berkus <josh@agliodbs.com>
@ 2003-09-12 16:57 ` Richard Jones <rj@last.fm>
0 siblings, 0 replies; 14+ messages in thread
From: Richard Jones @ 2003-09-12 16:57 UTC (permalink / raw)
To: pgsql-performance
The dual xeon arrangement is because the machine will also have to do some
collaborative filtering which is very cpu intensive and very disk
un-intensive, after loading the data into ram.
On Friday 12 September 2003 5:49 pm, you wrote:
> RIchard,
>
> > its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks
> >
> > should i mirror 2 of the disks for postgres data, and use the 3rd disk
> > for
>
> the
>
> > o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg
> > and use the 3rd for o/s,logs,backups ?
>
> I'd mirror 2. Stripey RAID with few disks imposes a heavy performance
> penalty on data writes (particularly updates), sometimes as much as 50% for
> a RAID5-3disk config.
>
> I am a little curious why you've got a dual-xeon, but could only afford 3
> disks ....
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance - Dell
@ 2003-09-12 17:03 Thom Dyson <TDyson@sybex.com>
2003-09-12 17:33 ` Re: best arrangement of 3 disks for (insert) performance Will LaShell <will@lashell.net>
2003-09-12 19:06 ` Re: best arrangement of 3 disks for (insert) performance - Dell Vivek Khera <khera@kcilink.com>
2003-09-12 20:47 ` Re: best arrangement of 3 disks for (insert) performance - Dell Christopher Browne <cbbrowne@libertyrms.info>
0 siblings, 3 replies; 14+ messages in thread
From: Thom Dyson @ 2003-09-12 17:03 UTC (permalink / raw)
To: pgsql-performance
The Dell PERC controllers have a very strong reputation for terrible
performance. If you search the archives of the Dell Linux Power Edge list
(dell.com/linux), you will find many, many people who get better
performance from software RAID, rather than the hw RAID on the PERC.
Having said that, the 3/SC might be one of the better PERC controllers. I
would spend and hour or two and benchmark hw vs. sw before I committed to
either one.
Thom Dyson
Director of Information Services
Sybex, Inc.
On 9/12/2003 9:55:40 AM, Richard Jones <rj@last.fm> wrote:
> The machine is coming from dell, and i have the option of a
> PERC 3/SC RAID Controller (32MB)
> or software raid.
>
> does anyone have any experience of this controller?
> its an additional £345 for this controller, i'd be interested to know
what
> people think - my other option is to buy the raid controller separately,
> which appeals to me but i wouldnt know what to look for in a raid
> controller.
>
> that raid controller review site sounds like a good idea :)
>
> Richard.
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 17:03 Re: best arrangement of 3 disks for (insert) performance - Dell Thom Dyson <TDyson@sybex.com>
@ 2003-09-12 17:33 ` Will LaShell <will@lashell.net>
2003-09-12 19:23 ` Re: best arrangement of 3 disks for (insert) performance Vivek Khera <khera@kcilink.com>
2 siblings, 1 reply; 14+ messages in thread
From: Will LaShell @ 2003-09-12 17:33 UTC (permalink / raw)
To: Thom Dyson <TDyson@sybex.com>; +Cc: pgsql-performance
I would like to point out though on the PERC controllers that are LSI
based ( Megaraid ) there -are- settings that can be changed to fix any
o the performance issues. Check the linux megaraid driver list archives
to see the full description. I've seen it come up many times and
basically all the problems have turned up resolved.
Will
On Fri, 2003-09-12 at 10:03, Thom Dyson wrote:
>
> The Dell PERC controllers have a very strong reputation for terrible
> performance. If you search the archives of the Dell Linux Power Edge list
> (dell.com/linux), you will find many, many people who get better
> performance from software RAID, rather than the hw RAID on the PERC.
> Having said that, the 3/SC might be one of the better PERC controllers. I
> would spend and hour or two and benchmark hw vs. sw before I committed to
> either one.
>
> Thom Dyson
> Director of Information Services
> Sybex, Inc.
>
> On 9/12/2003 9:55:40 AM, Richard Jones <rj@last.fm> wrote:
> > The machine is coming from dell, and i have the option of a
> > PERC 3/SC RAID Controller (32MB)
> > or software raid.
> >
> > does anyone have any experience of this controller?
> > its an additional £345 for this controller, i'd be interested to know
> what
> > people think - my other option is to buy the raid controller separately,
> > which appeals to me but i wouldnt know what to look for in a raid
> > controller.
> >
> > that raid controller review site sounds like a good idea :)
> >
> > Richard.
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
Attachments:
signature.asc (application/pgp-signature, 188B) download
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance
2003-09-12 17:03 Re: best arrangement of 3 disks for (insert) performance - Dell Thom Dyson <TDyson@sybex.com>
2003-09-12 17:33 ` Re: best arrangement of 3 disks for (insert) performance Will LaShell <will@lashell.net>
@ 2003-09-12 19:23 ` Vivek Khera <khera@kcilink.com>
0 siblings, 0 replies; 14+ messages in thread
From: Vivek Khera @ 2003-09-12 19:23 UTC (permalink / raw)
To: pgsql-performance
>>>>> "WL" == Will LaShell <will@lashell.net> writes:
WL> o the performance issues. Check the linux megaraid driver list archives
WL> to see the full description. I've seen it come up many times and
WL> basically all the problems have turned up resolved.
I've seen this advice a couple of times, but perhaps I'm just not a
good archive searcher because I can't find such recommendations on the
linux-megaraid-devel list archives...
Anyone have a direct pointer to right info?
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: khera@kciLink.com Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance - Dell
2003-09-12 17:03 Re: best arrangement of 3 disks for (insert) performance - Dell Thom Dyson <TDyson@sybex.com>
@ 2003-09-12 19:06 ` Vivek Khera <khera@kcilink.com>
2 siblings, 0 replies; 14+ messages in thread
From: Vivek Khera @ 2003-09-12 19:06 UTC (permalink / raw)
To: pgsql-performance
>>>>> "TD" == Thom Dyson <TDyson@sybex.com> writes:
TD> The Dell PERC controllers have a very strong reputation for terrible
TD> performance. If you search the archives of the Dell Linux Power Edge list
TD> (dell.com/linux), you will find many, many people who get better
TD> performance from software RAID, rather than the hw RAID on the PERC.
The PERC controllers are just a fancy name for a whole host of
different hardware. I have several, and some are made by LSI and some
are made by Adaptec. My latest is PERC3/DC which is an LSI MegaRAID
and is pretty darned fast.
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: khera@kciLink.com Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: best arrangement of 3 disks for (insert) performance - Dell
2003-09-12 17:03 Re: best arrangement of 3 disks for (insert) performance - Dell Thom Dyson <TDyson@sybex.com>
@ 2003-09-12 20:47 ` Christopher Browne <cbbrowne@libertyrms.info>
2 siblings, 0 replies; 14+ messages in thread
From: Christopher Browne @ 2003-09-12 20:47 UTC (permalink / raw)
To: pgsql-performance
TDyson@sybex.com ("Thom Dyson") writes:
> The Dell PERC controllers have a very strong reputation for terrible
> performance. If you search the archives of the Dell Linux Power
> Edge list (dell.com/linux), you will find many, many people who get
> better performance from software RAID, rather than the hw RAID on
> the PERC. Having said that, the 3/SC might be one of the better
> PERC controllers. I would spend and hour or two and benchmark hw
> vs. sw before I committed to either one.
I can't agree with that.
1. If you search the archives for messages dated a couple of years
ago, you can find lots of messages indicating terrible performance.
Drivers are not cast in concrete; there has been a LOT of change to
them since then.
2. The second MAJOR merit to hardware RAID is the ability to hot-swap
drives. Software RAID doesn't help with that at all.
3. The _immense_ performance improvement that can be gotten out of
these controllers comes from having fsync() turn into a near no-op
since changes can be committed to the 128K battery-backed cache REALLY
QUICKLY.
That is something you should avoid doing with software RAID in any
case where you actually care about your data.
That third part is where Big Wins come. It is the very same sort of
"big win from cacheing" that we saw, years ago, when we improved
system performance _immensely_ by adding a mere 16 bytes of cache by
buying serial controller cards with cacheing UUARTs. It is akin to
the way SCSI controllers got pretty big performance improvements by
adding 256 bytes of tagged command cache.
--
output = ("cbbrowne" "@" "libertyrms.info")
<http://dev6.int.libertyrms.com/;
Christopher Browne
(416) 646 3304 x124 (land)
^ permalink raw reply [nested|flat] 14+ messages in thread
end of thread, other threads:[~2003-09-15 08:05 UTC | newest]
Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-09-12 10:26 best arrangement of 3 disks for (insert) performance Richard Jones <rj@last.fm>
2003-09-12 11:06 ` Matt Clark <matt@ymogen.net>
2003-09-12 15:24 ` Christopher Browne <cbbrowne@libertyrms.info>
2003-09-12 16:55 ` Richard Jones <rj@last.fm>
2003-09-12 22:11 ` Rod Taylor <rbt@rbt.ca>
2003-09-14 04:44 ` Cott Lang <cott@internetstaff.com>
2003-09-15 08:05 ` Ron Johnson <ron.l.johnson@cox.net>
2003-09-12 16:49 ` Josh Berkus <josh@agliodbs.com>
2003-09-12 16:57 ` Richard Jones <rj@last.fm>
2003-09-12 17:03 Re: best arrangement of 3 disks for (insert) performance - Dell Thom Dyson <TDyson@sybex.com>
2003-09-12 17:33 ` Will LaShell <will@lashell.net>
2003-09-12 19:23 ` Vivek Khera <khera@kcilink.com>
2003-09-12 19:06 ` Re: best arrangement of 3 disks for (insert) performance - Dell Vivek Khera <khera@kcilink.com>
2003-09-12 20:47 ` Re: best arrangement of 3 disks for (insert) performance - Dell Christopher Browne <cbbrowne@libertyrms.info>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox