pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedBest OS for Postgres 8.2
38+ messages / 23 participants
[nested] [flat]
* Best OS for Postgres 8.2
@ 2007-05-07 21:55 David Levy <dvid.levy@gmail.com>
2007-05-07 21:59 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 00:40 ` Re: Best OS for Postgres 8.2 Chris <dmagick@gmail.com>
2007-05-08 03:41 ` Re: Best OS for Postgres 8.2 Ron <rjpeace@earthlink.net>
2007-05-08 03:56 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
0 siblings, 5 replies; 38+ messages in thread
From: David Levy @ 2007-05-07 21:55 UTC (permalink / raw)
To: pgsql-performance
Hi,
I am about to order a new server for my Postgres cluster. I will
probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
Which OS would you recommend to optimize Postgres behaviour (i/o
access, multithreading, etc) ?
I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
help with this ?
Regards
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
@ 2007-05-07 21:59 ` Joshua D. Drake <jd@commandprompt.com>
2007-05-07 22:04 ` Re: Best OS for Postgres 8.2 Bill Moran <wmoran@collaborativefusion.com>
4 siblings, 1 reply; 38+ messages in thread
From: Joshua D. Drake @ 2007-05-07 21:59 UTC (permalink / raw)
To: David Levy <dvid.levy@gmail.com>; +Cc: pgsql-performance
David Levy wrote:
> Hi,
>
> I am about to order a new server for my Postgres cluster. I will
> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
> Which OS would you recommend to optimize Postgres behaviour (i/o
> access, multithreading, etc) ?
>
> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
> help with this ?
Well you just described three linux distributions, which is hardly a
question about which OS to use ;). I would stick with the long supported
versions of Linux, thus CentOS 5, Debian 4, Ubuntu Dapper.
Joshua D. Drake
>
>
> Regards
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 21:59 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
@ 2007-05-07 22:04 ` Bill Moran <wmoran@collaborativefusion.com>
2007-05-07 22:14 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
0 siblings, 1 reply; 38+ messages in thread
From: Bill Moran @ 2007-05-07 22:04 UTC (permalink / raw)
To: Joshua D. Drake <jd@commandprompt.com>; +Cc: David Levy <dvid.levy@gmail.com>; pgsql-performance
In response to "Joshua D. Drake" <jd@commandprompt.com>:
> David Levy wrote:
> > Hi,
> >
> > I am about to order a new server for my Postgres cluster. I will
> > probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
> > Which OS would you recommend to optimize Postgres behaviour (i/o
> > access, multithreading, etc) ?
> >
> > I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
> > help with this ?
>
> Well you just described three linux distributions, which is hardly a
> question about which OS to use ;). I would stick with the long supported
> versions of Linux, thus CentOS 5, Debian 4, Ubuntu Dapper.
There used to be a prominent site that recommended FreeBSD for Postgres.
Don't know if that's still recommended or not -- but bringing it up is
likely to start a Holy War.
--
Bill Moran
Collaborative Fusion Inc.
http://people.collaborativefusion.com/~wmoran/
wmoran@collaborativefusion.com
Phone: 412-422-3463x4023
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 21:59 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
2007-05-07 22:04 ` Re: Best OS for Postgres 8.2 Bill Moran <wmoran@collaborativefusion.com>
@ 2007-05-07 22:14 ` Joshua D. Drake <jd@commandprompt.com>
2007-05-08 09:48 ` Re: Best OS for Postgres 8.2 Steinar H. Gunderson <sgunderson@bigfoot.com>
0 siblings, 1 reply; 38+ messages in thread
From: Joshua D. Drake @ 2007-05-07 22:14 UTC (permalink / raw)
To: Bill Moran <wmoran@collaborativefusion.com>; +Cc: David Levy <dvid.levy@gmail.com>; pgsql-performance
Bill Moran wrote:
> In response to "Joshua D. Drake" <jd@commandprompt.com>:
>
>> David Levy wrote:
>>> Hi,
>>>
>>> I am about to order a new server for my Postgres cluster. I will
>>> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>>> Which OS would you recommend to optimize Postgres behaviour (i/o
>>> access, multithreading, etc) ?
>>>
>>> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>>> help with this ?
>> Well you just described three linux distributions, which is hardly a
>> question about which OS to use ;). I would stick with the long supported
>> versions of Linux, thus CentOS 5, Debian 4, Ubuntu Dapper.
>
> There used to be a prominent site that recommended FreeBSD for Postgres.
> Don't know if that's still recommended or not -- but bringing it up is
> likely to start a Holy War.
Heh... I doubt it will start a war. FreeBSD is a good OS. However, I
specifically noted the Dual Xeon Quad Core, which means, 8 procs. It is
my understanding (and I certainly could be wrong) that FreeBSD doesn't
handle SMP nearly as well as Linux (and Linux not as well as Solaris).
Sincerely,
Joshua D. Drake
>
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 21:59 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
2007-05-07 22:04 ` Re: Best OS for Postgres 8.2 Bill Moran <wmoran@collaborativefusion.com>
2007-05-07 22:14 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
@ 2007-05-08 09:48 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2007-05-08 10:27 ` Re: Best OS for Postgres 8.2 david@lang.hm
0 siblings, 1 reply; 38+ messages in thread
From: Steinar H. Gunderson @ 2007-05-08 09:48 UTC (permalink / raw)
To: pgsql-performance
On Mon, May 07, 2007 at 03:14:08PM -0700, Joshua D. Drake wrote:
> It is my understanding (and I certainly could be wrong) that FreeBSD
> doesn't handle SMP nearly as well as Linux (and Linux not as well as
> Solaris).
I'm not actually sure about the last part. There are installations as big as
1024 CPUs that run Linux -- most people won't need that, but it's probably an
indicator that eight cores should run OK :-)
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 21:59 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
2007-05-07 22:04 ` Re: Best OS for Postgres 8.2 Bill Moran <wmoran@collaborativefusion.com>
2007-05-07 22:14 ` Re: Best OS for Postgres 8.2 Joshua D. Drake <jd@commandprompt.com>
2007-05-08 09:48 ` Re: Best OS for Postgres 8.2 Steinar H. Gunderson <sgunderson@bigfoot.com>
@ 2007-05-08 10:27 ` david@lang.hm
0 siblings, 0 replies; 38+ messages in thread
From: david@lang.hm @ 2007-05-08 10:27 UTC (permalink / raw)
To: Steinar H. Gunderson <sgunderson@bigfoot.com>; +Cc: pgsql-performance
On Tue, 8 May 2007, Steinar H. Gunderson wrote:
> On Mon, May 07, 2007 at 03:14:08PM -0700, Joshua D. Drake wrote:
>> It is my understanding (and I certainly could be wrong) that FreeBSD
>> doesn't handle SMP nearly as well as Linux (and Linux not as well as
>> Solaris).
>
> I'm not actually sure about the last part. There are installations as big as
> 1024 CPUs that run Linux -- most people won't need that, but it's probably an
> indicator that eight cores should run OK :-)
over the weekend the question of scalability was raised on the linux
kernel mailing list and people are shipping 1024 cpu systems with linux,
and testing 4096 cpu systems. there are occasionally still bottlenecks
that limit scalability, butunless you run into a bad driver or filesystem
you should have no problems in the 8-16 core range.
any comparison between Linux and any other OS needs to include a date for
when the comparison was made, Linux is changing at a frightning pace (I
think I saw something within the last few weeks that said that the rate of
change for the kernel has averaged around 9000 lines of code per day over
the last couple of years) you need to re-check comparisons every year or
two or you end up working with obsolete data.
David Lang
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
@ 2007-05-07 22:57 ` Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
4 siblings, 1 reply; 38+ messages in thread
From: Steve Atkins @ 2007-05-07 22:57 UTC (permalink / raw)
To: pgsql-performance
On May 7, 2007, at 2:55 PM, David Levy wrote:
> Hi,
>
> I am about to order a new server for my Postgres cluster. I will
> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
> Which OS would you recommend to optimize Postgres behaviour (i/o
> access, multithreading, etc) ?
>
> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
> help with this ?
Well, all three you mention are much the same, just with a different
badge on the box, as far as performance is concerned. They're all
going to be a moderately recent Linux kernel, with your choice
of filesystems, so any choice between them is going to be driven
more by available staff and support or personal preference.
I'd probably go CentOS 5 over Fedora just because Fedora doesn't
get supported for very long - more of an issue with a dedicated
database box with a long lifespan than your typical desktop or
interchangeable webserver.
I might also look at Solaris 10, though. I've yet to play with it
much, but it
seems nice, and I suspect it might manage 8 cores better than current
Linux setups.
Cheers,
Steve
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
@ 2007-05-08 04:41 ` 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
0 siblings, 1 reply; 38+ messages in thread
From: 李彦 Ian Li @ 2007-05-08 04:41 UTC (permalink / raw)
To: pgsql-performance
In #postgresql on freenode, somebody ever mentioned that ZFS from
Solaris helps a lot to the performance of pgsql, so dose anyone have
information about that?
Steve Atkins wrote:
>
> On May 7, 2007, at 2:55 PM, David Levy wrote:
>
>> Hi,
>>
>> I am about to order a new server for my Postgres cluster. I will
>> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>> Which OS would you recommend to optimize Postgres behaviour (i/o
>> access, multithreading, etc) ?
>>
>> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>> help with this ?
>
> Well, all three you mention are much the same, just with a different
> badge on the box, as far as performance is concerned. They're all
> going to be a moderately recent Linux kernel, with your choice
> of filesystems, so any choice between them is going to be driven
> more by available staff and support or personal preference.
>
> I'd probably go CentOS 5 over Fedora just because Fedora doesn't
> get supported for very long - more of an issue with a dedicated
> database box with a long lifespan than your typical desktop or
> interchangeable webserver.
>
> I might also look at Solaris 10, though. I've yet to play with it much,
> but it
> seems nice, and I suspect it might manage 8 cores better than current
> Linux setups.
>
> Cheers,
> Steve
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>
Regards
Ian
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
@ 2007-05-08 07:59 ` david@lang.hm
2007-05-08 08:21 ` Re: Best OS for Postgres 8.2 Heikki Linnakangas <heikki@enterprisedb.com>
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 09:43 ` Re: Best OS for Postgres 8.2 Alexander Staubo <alex@purefiction.net>
2007-05-08 16:58 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 20:49 ` Re: Best OS for Postgres 8.2 Charles Sprickman <spork@bway.net>
2007-05-09 17:10 ` Re: Best OS for Postgres 8.2 Jim Nasby <decibel@decibel.org>
0 siblings, 6 replies; 38+ messages in thread
From: david@lang.hm @ 2007-05-08 07:59 UTC (permalink / raw)
To: 李彦 Ian Li <liyan82@gmail.com>; +Cc: pgsql-performance
On Tue, 8 May 2007, æ~]~N彦 Ian Li wrote:
> In #postgresql on freenode, somebody ever mentioned that ZFS from Solaris
> helps a lot to the performance of pgsql, so dose anyone have information
> about that?
the filesystem you use will affect the performance of postgres
significantly. I've heard a lot of claims for ZFS, unfortunantly many of
them from people who have prooven that they didn't know what they were
talking about by the end of their first or second e-mails.
much of the hype for ZFS is it's volume management capabilities and admin
tools. Linux has most (if not all) of the volume management capabilities,
it just seperates them from the filesystems so that any filesystem can use
them, and as a result you use one tool to setup your RAID, one to setup
snapshots, and a third to format your filesystems where ZFS does this in
one userspace tool.
once you seperate the volume management piece out, the actual performance
question is a lot harder to answer. there are a lot of people who say that
it's far faster then the alternate filesystems on Solaris, but I haven't
seen any good comparisons between it and Linux filesystems.
On Linux you have the choice of several filesystems, and the perfomance
will vary wildly depending on your workload. I personally tend to favor
ext2 (for small filesystems where the application is ensuring data
integrity) or XFS (for large filesystems)
I personally don't trust reiserfs, jfs seems to be a tools for
transitioning from AIX more then anything else, and ext3 seems to have all
the scaling issues of ext2 plus the overhead (and bottleneck) of
journaling.
one issue with journaling filesystems, if you journal the data as well as
the metadata you end up with a very reliable setup, however it means that
all your data needs to be written twice, oncce to the journal, and once to
the final location. the write to the journal can be slightly faster then a
normal write to the final location (the journal is a sequential write to
an existing file), however the need to write twice can effectivly cut your
disk I/O bandwidth in half when doing heavy writes. worse, when you end up
writing mor ethen will fit in the journal (128M is the max for ext3) the
entire system then needs to stall while the journal gets cleared to make
space for the additional writes.
if you don't journal your data then you avoid the problems above, but in a
crash you may find that you lost data, even though the filesystem is
'intact' according to fsck.
David Lang
> Steve Atkins wrote:
>>
>> On May 7, 2007, at 2:55 PM, David Levy wrote:
>>
>> > Hi,
>> >
>> > I am about to order a new server for my Postgres cluster. I will
>> > probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>> > Which OS would you recommend to optimize Postgres behaviour (i/o
>> > access, multithreading, etc) ?
>> >
>> > I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>> > help with this ?
>>
>> Well, all three you mention are much the same, just with a different
>> badge on the box, as far as performance is concerned. They're all
>> going to be a moderately recent Linux kernel, with your choice
>> of filesystems, so any choice between them is going to be driven
>> more by available staff and support or personal preference.
>>
>> I'd probably go CentOS 5 over Fedora just because Fedora doesn't
>> get supported for very long - more of an issue with a dedicated
>> database box with a long lifespan than your typical desktop or
>> interchangeable webserver.
>>
>> I might also look at Solaris 10, though. I've yet to play with it much,
>> but it
>> seems nice, and I suspect it might manage 8 cores better than current
>> Linux setups.
>>
>> Cheers,
>> Steve
>>
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 5: don't forget to increase your free space map settings
>>
>
> Regards
>
> Ian
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
From pgsql-performance-owner@postgresql.org Tue May 8 05:09:47 2007
Received: from localhost (maia-2.hub.org [200.46.204.187])
by postgresql.org (Postfix) with ESMTP id B7F2B9FBB48
for <pgsql-performance-postgresql.org@postgresql.org>; Tue, 8 May 2007 05:09:46 -0300 (ADT)
Received: from postgresql.org ([200.46.204.71])
by localhost (mx1.hub.org [200.46.204.187]) (amavisd-maia, port 10024)
with ESMTP id 21379-08 for <pgsql-performance-postgresql.org@postgresql.org>;
Tue, 8 May 2007 05:09:40 -0300 (ADT)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.4
Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.234])
by postgresql.org (Postfix) with ESMTP id C1AE49FBB21
for <pgsql-performance@postgresql.org>; Tue, 8 May 2007 05:09:42 -0300 (ADT)
Received: by nz-out-0506.google.com with SMTP id s1so1988964nze
for <pgsql-performance@postgresql.org>; Tue, 08 May 2007 01:09:41 -0700 (PDT)
DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed;
d=gmail.com; s=beta;
h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
b=tMnqJ4jJFkANPEKsjO48H7tdlMkh1PtxD9ojia3Hs7kK4jLUtNUWSWo6GEPQw3nHKK/IFUU42X/xGazATgo19i5FXATTGs/NxHYcWcT4jYq/r84Fmsj2ndDu3lnjOLK8pGaJ+Di1Rw1oLfhcu+zCkBrfTX9KTtbO4BSU2riTwHc=
DomainKey-Signature: a=rsa-sha1; c=nofws;
d=gmail.com; s=beta;
h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
b=ibMOoo+Gg9CIe5nVV5w4I8kJBzehuai5dBAmssoI98av8cCWzpmlj29ellDiBihXGB3g1EbKZR/mldl4oZV3yA/kvjUtA5mAHvvxnkQMjMHzEOOokHx/41kpWk6p14IjD8PYoLxlcaggO4I9y9xyJR8/1+ikTKvqdLOk3eUa+tc=
Received: by 10.115.95.1 with SMTP id x1mr2497915wal.1178611781182;
Tue, 08 May 2007 01:09:41 -0700 (PDT)
Received: by 10.114.160.9 with HTTP; Tue, 8 May 2007 01:09:40 -0700 (PDT)
Message-ID: <b41c75520705080109g7655f5f2m5592595c9a578c16@mail.gmail.com>
Date: Tue, 8 May 2007 10:09:40 +0200
From: "Claus Guttesen" <kometen@gmail.com>
To: "David Levy" <dvid.levy@gmail.com>
Subject: Re: Best OS for Postgres 8.2
Cc: pgsql-performance@postgresql.org
In-Reply-To: <1178574927.533849.90730@e51g2000hsg.googlegroups.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <1178574927.533849.90730@e51g2000hsg.googlegroups.com>
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200705/86
X-Sequence-Number: 24476
> I am about to order a new server for my Postgres cluster. I will
> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
> Which OS would you recommend to optimize Postgres behaviour (i/o
> access, multithreading, etc) ?
>
> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
> help with this ?
My only experience is with FreeBSD. My installation is running 6.2 and
pg 7.4 on a four-way woodcrest and besides being very stable it's also
performing very well. But then FreeBSD 6.x might not scale as well
beyond four cores atm. There you probably would need FreeBSD 7 which
is the development branch and should require extensive testing.
How big will the db be in size?
--
regards
Claus
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 08:21 ` Heikki Linnakangas <heikki@enterprisedb.com>
5 siblings, 0 replies; 38+ messages in thread
From: Heikki Linnakangas @ 2007-05-08 08:21 UTC (permalink / raw)
To: david@lang.hm; +Cc: 李彦 Ian Li <liyan82@gmail.com>; pgsql-performance
david@lang.hm wrote:
> if you don't journal your data then you avoid the problems above, but in
> a crash you may find that you lost data, even though the filesystem is
> 'intact' according to fsck.
PostgreSQL itself journals it's data to the WAL, so that shouldn't happen.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 08:22 ` Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
5 siblings, 1 reply; 38+ messages in thread
From: Claus Guttesen @ 2007-05-08 08:22 UTC (permalink / raw)
To: david@lang.hm <david@lang.hm>; +Cc: 李彦 Ian Li <liyan82@gmail.com>; pgsql-performance
> > In #postgresql on freenode, somebody ever mentioned that ZFS from Solaris
> > helps a lot to the performance of pgsql, so dose anyone have information
> > about that?
>
> the filesystem you use will affect the performance of postgres
> significantly. I've heard a lot of claims for ZFS, unfortunantly many of
> them from people who have prooven that they didn't know what they were
> talking about by the end of their first or second e-mails.
>
> much of the hype for ZFS is it's volume management capabilities and admin
> tools. Linux has most (if not all) of the volume management capabilities,
> it just seperates them from the filesystems so that any filesystem can use
> them, and as a result you use one tool to setup your RAID, one to setup
> snapshots, and a third to format your filesystems where ZFS does this in
> one userspace tool.
Even though those posters may have proven them selves wrong, zfs is
still a very handy fs and it should not be judged relative to these
statements.
> once you seperate the volume management piece out, the actual performance
> question is a lot harder to answer. there are a lot of people who say that
> it's far faster then the alternate filesystems on Solaris, but I haven't
> seen any good comparisons between it and Linux filesystems.
One could install pg on solaris 10 and format the data-area as ufs and
then as zfs and compare import- and query-times and other benchmarking
but comparing ufs/zfs to Linux-filesystems would also be a comparison
of those two os'es.
--
regards
Claus
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
@ 2007-05-08 08:45 ` david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-11 13:35 ` Re: Best OS for Postgres 8.2 Adam Witney <awitney@sgul.ac.uk>
0 siblings, 2 replies; 38+ messages in thread
From: david@lang.hm @ 2007-05-08 08:45 UTC (permalink / raw)
To: Claus Guttesen <kometen@gmail.com>; +Cc: 李彦 Ian Li <liyan82@gmail.com>; pgsql-performance
On Tue, 8 May 2007, Claus Guttesen wrote:
>> > In #postgresql on freenode, somebody ever mentioned that ZFS from
>> > Solaris
>> > helps a lot to the performance of pgsql, so dose anyone have information
>> > about that?
>>
>> the filesystem you use will affect the performance of postgres
>> significantly. I've heard a lot of claims for ZFS, unfortunantly many of
>> them from people who have prooven that they didn't know what they were
>> talking about by the end of their first or second e-mails.
>>
>> much of the hype for ZFS is it's volume management capabilities and admin
>> tools. Linux has most (if not all) of the volume management capabilities,
>> it just seperates them from the filesystems so that any filesystem can use
>> them, and as a result you use one tool to setup your RAID, one to setup
>> snapshots, and a third to format your filesystems where ZFS does this in
>> one userspace tool.
>
> Even though those posters may have proven them selves wrong, zfs is
> still a very handy fs and it should not be judged relative to these
> statements.
I don't disagree with you, I'm just noteing that too many of the 'ZFS is
great' posts need to be discounted as a result (the same thing goes for
the 'reiserfs4 is great' posts)
>> once you seperate the volume management piece out, the actual performance
>> question is a lot harder to answer. there are a lot of people who say that
>> it's far faster then the alternate filesystems on Solaris, but I haven't
>> seen any good comparisons between it and Linux filesystems.
>
> One could install pg on solaris 10 and format the data-area as ufs and
> then as zfs and compare import- and query-times and other benchmarking
> but comparing ufs/zfs to Linux-filesystems would also be a comparison
> of those two os'es.
however, such a comparison is very legitimate, it doesn't really matter
which filesystem is better if the OS that it's tied to limits it so much
that the other one wins out with an inferior filesystem
currently ZFS is only available on Solaris, parts of it have been released
under GPLv2, but it doesn't look like enough of it to be ported to Linux
(enough was released for grub to be able to access it read-only, but not
the full filesystem). there are also patent concerns that are preventing
any porting to Linux.
on the other hand, it's integrated userspace tools are pushing people to
create similar tools for Linux (without needeing to combine the vairous
pieces in the kernel)
David Lang
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 08:49 ` Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
1 sibling, 1 reply; 38+ messages in thread
From: Trygve Laugstøl @ 2007-05-08 08:49 UTC (permalink / raw)
To: pgsql-performance
david@lang.hm wrote:
> On Tue, 8 May 2007, Claus Guttesen wrote:
>
>>> > In #postgresql on freenode, somebody ever mentioned that ZFS from
>>> > Solaris
>>> > helps a lot to the performance of pgsql, so dose anyone have
>>> information
>>> > about that?
>>>
>>> the filesystem you use will affect the performance of postgres
>>> significantly. I've heard a lot of claims for ZFS, unfortunantly
>>> many of
>>> them from people who have prooven that they didn't know what they were
>>> talking about by the end of their first or second e-mails.
>>>
>>> much of the hype for ZFS is it's volume management capabilities and
>>> admin
>>> tools. Linux has most (if not all) of the volume management
>>> capabilities,
>>> it just seperates them from the filesystems so that any filesystem
>>> can use
>>> them, and as a result you use one tool to setup your RAID, one to setup
>>> snapshots, and a third to format your filesystems where ZFS does
>>> this in
>>> one userspace tool.
>>
>> Even though those posters may have proven them selves wrong, zfs is
>> still a very handy fs and it should not be judged relative to these
>> statements.
>
> I don't disagree with you, I'm just noteing that too many of the 'ZFS is
> great' posts need to be discounted as a result (the same thing goes for
> the 'reiserfs4 is great' posts)
>
>>> once you seperate the volume management piece out, the actual
>>> performance
>>> question is a lot harder to answer. there are a lot of people who
>>> say that
>>> it's far faster then the alternate filesystems on Solaris, but I
>>> haven't
>>> seen any good comparisons between it and Linux filesystems.
>>
>> One could install pg on solaris 10 and format the data-area as ufs and
>> then as zfs and compare import- and query-times and other benchmarking
>> but comparing ufs/zfs to Linux-filesystems would also be a comparison
>> of those two os'es.
>
> however, such a comparison is very legitimate, it doesn't really matter
> which filesystem is better if the OS that it's tied to limits it so much
> that the other one wins out with an inferior filesystem
>
> currently ZFS is only available on Solaris, parts of it have been
> released under GPLv2, but it doesn't look like enough of it to be ported
> to Linux (enough was released for grub to be able to access it
> read-only, but not the full filesystem). there are also patent concerns
> that are preventing any porting to Linux.
This is not entirely correct. ZFS is only under the CDDL license and it
has been ported to FreeBSD.
http://mail.opensolaris.org/pipermail/zfs-discuss/2007-April/026922.html
--
Trygve
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
@ 2007-05-08 10:16 ` david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
0 siblings, 1 reply; 38+ messages in thread
From: david@lang.hm @ 2007-05-08 10:16 UTC (permalink / raw)
To: Trygve Laugstøl <trygvis@inamo.no>; +Cc: pgsql-performance
On Tue, 8 May 2007, Trygve Laugstøl wrote:
>> currently ZFS is only available on Solaris, parts of it have been released
>> under GPLv2, but it doesn't look like enough of it to be ported to Linux
>> (enough was released for grub to be able to access it read-only, but not
>> the full filesystem). there are also patent concerns that are preventing
>> any porting to Linux.
>
> This is not entirely correct. ZFS is only under the CDDL license and it has
> been ported to FreeBSD.
>
> http://mail.opensolaris.org/pipermail/zfs-discuss/2007-April/026922.html
>
I wonder how they handled the license issues? I thought that if you
combined stuff that was BSD licensed with stuff with a more restrictive
license the result was under the more restrictive license. thanks for the
info.
here's a link about the GPLv2 stuff for zfs
http://blogs.sun.com/darren/entry/zfs_under_gplv2_already_exists
From pgsql-performance-owner@postgresql.org Tue May 8 07:22:31 2007
Received: from localhost (maia-4.hub.org [200.46.204.183])
by postgresql.org (Postfix) with ESMTP id 3ED559FB28A
for <pgsql-performance-postgresql.org@postgresql.org>; Tue, 8 May 2007 07:22:29 -0300 (ADT)
Received: from postgresql.org ([200.46.204.71])
by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024)
with ESMTP id 58273-05 for <pgsql-performance-postgresql.org@postgresql.org>;
Tue, 8 May 2007 07:22:26 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.5
Received: from oxford.xeocode.com (unknown [62.232.55.118])
by postgresql.org (Postfix) with ESMTP id 1ACC39FB1B7
for <pgsql-performance@postgresql.org>; Tue, 8 May 2007 07:22:26 -0300 (ADT)
Received: from localhost ([127.0.0.1] helo=oxford.xeocode.com)
by oxford.xeocode.com with esmtp (Exim 4.67)
(envelope-from <stark@enterprisedb.com>)
id 1HlMpr-00010j-V3; Tue, 08 May 2007 11:22:24 +0100
From: Gregory Stark <stark@enterprisedb.com>
To: "Pomarede Nicolas" <npomarede@corp.free.fr>
Cc: <pgsql-performance@postgresql.org>
Subject: Re: truncate a table instead of vaccum full when count(*) is 0
In-Reply-To: <Pine.LNX.4.64.0705081125140.20675@localhost> (Pomarede Nicolas's
message of "Tue, 8 May 2007 11:43:14 +0200 (CEST)")
Organization: EnterpriseDB
References: <Pine.LNX.4.64.0705081125140.20675@localhost>
X-Draft-From: ("nnimap+mail01.enterprisedb.com:INBOX.performance" 216)
Date: Tue, 08 May 2007 11:22:23 +0100
Message-ID: <87d51b1uow.fsf@oxford.xeocode.com>
User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200705/100
X-Sequence-Number: 24490
"Pomarede Nicolas" <npomarede@corp.free.fr> writes:
> But for the data (dead rows), even running a vacuum analyze every day is not
> enough, and doesn't truncate some empty pages at the end, so the data size
> remains in the order of 200-300 MB, when only a few effective rows are there.
Try running vacuum more frequently. Once per day isn't very frequent for
vacuum, every 60 or 30 minutes isn't uncommon. For your situation you might
even consider running it continuously in a loop.
> I see in the 8.3 list of coming changes that the FSM will try to re-use pages
> in a better way to help truncating empty pages. Is this correct ?
There are several people working on improvements to vacuum but it's not clear
right now exactly what we'll end up with. I think most of the directly vacuum
related changes wouldn't actually help you either.
The one that would help you is named "HOT". If you're interested in
experimenting with an experimental patch you could consider taking CVS and
applying HOT and seeing how it affects you. Or if you see an announcement that
it's been comitted taking a beta and experimenting with it before the 8.3
release could be interesting. Experiments with real-world databases can be
very helpful for developers since it's hard to construct truly realistic
benchmarks.
> So, I would like to truncate the table when the number of rows reaches 0 (just
> after the table was processed, and just before some new rows are added).
>
> Is there an easy way to do this under psql ? For example, lock the table, do a
> count(*), if result is 0 row then truncate the table, unlock the table (a kind
> of atomic 'truncate table if count(*) == 0').
>
> Would this work and what would be the steps ?
It would work but you may end up keeping the lock for longer than you're happy
for. Another option to consider would be to use CLUSTER instead of vacuum full
though the 8.2 CLUSTER wasn't entirely MVCC safe and I think in your situation
that might actually be a problem. It would cause transactions that started
before the cluster (but didn't access the table before the cluster) to not see
any records after the cluster.
--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 10:29 ` Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:55 ` Re: [OT] Best OS for Postgres 8.2 "C. Bergström" <cbergstrom@netsyncro.com>
2007-05-08 12:01 ` Re: Best OS for Postgres 8.2 Luke Lonergan <LLonergan@greenplum.com>
0 siblings, 2 replies; 38+ messages in thread
From: Trygve Laugstøl @ 2007-05-08 10:29 UTC (permalink / raw)
To: pgsql-performance
david@lang.hm wrote:
> On Tue, 8 May 2007, Trygve Laugstøl wrote:
>
>>> currently ZFS is only available on Solaris, parts of it have been
>>> released
>>> under GPLv2, but it doesn't look like enough of it to be ported to
>>> Linux
>>> (enough was released for grub to be able to access it read-only, but
>>> not
>>> the full filesystem). there are also patent concerns that are
>>> preventing
>>> any porting to Linux.
>>
>> This is not entirely correct. ZFS is only under the CDDL license and
>> it has been ported to FreeBSD.
>>
>> http://mail.opensolaris.org/pipermail/zfs-discuss/2007-April/026922.html
>>
> I wonder how they handled the license issues? I thought that if you
> combined stuff that was BSD licensed with stuff with a more restrictive
> license the result was under the more restrictive license. thanks for
> the info.
The CDDL is not a restrictive license like GPL, it is based on the MIT
license so it can be used with BSD stuff without problems. There are
lots of discussion going on (read: flamewars) on the opensolaris lists
about how it can/should it/will it be integrated into linux.
> here's a link about the GPLv2 stuff for zfs
>
> http://blogs.sun.com/darren/entry/zfs_under_gplv2_already_exists
That title is fairly misleading as it's only some read-only bits to be
able to boot off ZFS with grub.
--
Trygve
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: [OT] Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
@ 2007-05-08 10:55 ` "C. Bergström" <cbergstrom@netsyncro.com>
2007-05-08 12:25 ` Re: [OT] Best OS for Postgres 8.2 Adam Tauno Williams <adamtaunowilliams@gmail.com>
1 sibling, 1 reply; 38+ messages in thread
From: "C. Bergström" @ 2007-05-08 10:55 UTC (permalink / raw)
To: ; +Cc: pgsql-performance
I'm really not a senior member around here and while all this licensing
stuff and underlying fs between OSs is very interesting can we please
think twice before continuing it.
Thanks for the minute,
./C
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: [OT] Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:55 ` Re: [OT] Best OS for Postgres 8.2 "C. Bergström" <cbergstrom@netsyncro.com>
@ 2007-05-08 12:25 ` Adam Tauno Williams <adamtaunowilliams@gmail.com>
0 siblings, 0 replies; 38+ messages in thread
From: Adam Tauno Williams @ 2007-05-08 12:25 UTC (permalink / raw)
To: pgsql-performance
> I'm really not a senior member around here and while all this licensing
> stuff and underlying fs between OSs is very interesting can we please
> think twice before continuing it.
Agree, there are other lists for this stuff; and back to what one of
the original posters said: it doesn't matter much.
[Also not a regular poster, but I always gain something from reading
this list.]
Most people who really go into OS selection / FS selection are looking
for a cheap/silver bullet for performance. No such thing exists. The
difference made by any modern OS/FS is almost immaterial. You need to
do the slow slogging work of site/application specific optimization and
tuning; that is where you will find significant performance
improvements.
-
Adam Tauno Williams, Network & Systems Administrator
Consultant - http://www.whitemiceconsulting.com
Developer - http://www.opengroupware.org
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
@ 2007-05-08 12:01 ` Luke Lonergan <LLonergan@greenplum.com>
2007-05-08 16:08 ` Re: Best OS for Postgres 8.2 Ron <rjpeace@earthlink.net>
2007-05-09 03:51 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
1 sibling, 2 replies; 38+ messages in thread
From: Luke Lonergan @ 2007-05-08 12:01 UTC (permalink / raw)
To: Trygve Laugstøl <trygvis@inamo.no>; pgsql-performance
WRT ZFS on Linux, if someone were to port it, the license issue would get worked out IMO (with some discussion to back me up). From discussions with the developers, the biggest issue is a technical one: the Linux VFS layer makes the port difficult.
I don't hold any hope that the FUSE port will be a happy thing, the performance won't be there.
Any volunteers to port ZFS to Linux?
- Luke
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 12:01 ` Re: Best OS for Postgres 8.2 Luke Lonergan <LLonergan@greenplum.com>
@ 2007-05-08 16:08 ` Ron <rjpeace@earthlink.net>
1 sibling, 0 replies; 38+ messages in thread
From: Ron @ 2007-05-08 16:08 UTC (permalink / raw)
To: Luke Lonergan <LLonergan@greenplum.com>; pgsql-performance
I've seen the FUSE port of ZFS, and it does run sslloowwllyy. It
appears that a native linux port is going to be required if we want
ZFS to be reasonably performant.
WRT which FS to use for pg; the biggest issue is what kind of DB you
will be building. The best pg FS for OLTP and OLAP are not the same
IME. Ditto a dependence on how large your records and the amount of
IO in your typical transactions are.
For lot's of big, more reads than writes transactions, SGI's XFS
seems to be best.
XFS is not the best for OLTP. Especially for OLTP involving lots of small IOs.
jfs seems to be best for that.
Caveat: I have not yet experimented with any version of reiserfs in
production.
Cheers,
Ron Peacetree
At 08:01 AM 5/8/2007, Luke Lonergan wrote:
>WRT ZFS on Linux, if someone were to port it, the license issue
>would get worked out IMO (with some discussion to back me up). From
>discussions with the developers, the biggest issue is a technical
>one: the Linux VFS layer makes the port difficult.
>
>I don't hold any hope that the FUSE port will be a happy thing, the
>performance won't be there.
>
>Any volunteers to port ZFS to Linux?
>
>- Luke
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: don't forget to increase your free space map settings
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 12:01 ` Re: Best OS for Postgres 8.2 Luke Lonergan <LLonergan@greenplum.com>
@ 2007-05-09 03:51 ` Greg Smith <gsmith@gregsmith.com>
2007-05-09 08:57 ` Re: Best OS for Postgres 8.2 david@lang.hm
1 sibling, 1 reply; 38+ messages in thread
From: Greg Smith @ 2007-05-09 03:51 UTC (permalink / raw)
To: pgsql-performance
On Tue, 8 May 2007, Luke Lonergan wrote:
> From discussions with the developers, the biggest issue is a technical
> one: the Linux VFS layer makes the [ZFS] port difficult.
Difficult on two levels. First you'd have to figure out how to make it
work at all; then you'd have to reshape it into a form that it would be
acceptable to the Linux kernel developers, who haven't seemed real keen on
the idea so far.
The standard article I'm you've already seen this week on this topic is
Jeff Bonwick's at
http://blogs.sun.com/bonwick/entry/rampant_layering_violation
What really bugged me was his earlier article linked to there where he
talks about how ZFS eliminates the need for hardware RAID controllers:
http://blogs.sun.com/bonwick/entry/raid_z
While there may be merit to that idea for some applications, like
situations where you have a pig of a RAID5 volume, that's just hype for
database writes. "We issue the SYNCHRONIZE CACHE command to the disks
after pushing all data in a transaction group"--see, that would be the
part the hardware controller is needed to accelerate. If you really care
about whether your data hit disk, there is no way to break the RPM barrier
without hardware support. The fact that he misunderstands such a
fundamental point makes me wonder what other gigantic mistakes might be
buried in his analysis.
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 12:01 ` Re: Best OS for Postgres 8.2 Luke Lonergan <LLonergan@greenplum.com>
2007-05-09 03:51 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
@ 2007-05-09 08:57 ` david@lang.hm
2007-05-09 09:38 ` Re: Best OS for Postgres 8.2 Steinar H. Gunderson <sgunderson@bigfoot.com>
0 siblings, 1 reply; 38+ messages in thread
From: david@lang.hm @ 2007-05-09 08:57 UTC (permalink / raw)
To: Greg Smith <gsmith@gregsmith.com>; +Cc: pgsql-performance
On Tue, 8 May 2007, Greg Smith wrote:
> On Tue, 8 May 2007, Luke Lonergan wrote:
>
>> From discussions with the developers, the biggest issue is a technical
>> one: the Linux VFS layer makes the [ZFS] port difficult.
>
> Difficult on two levels. First you'd have to figure out how to make it work
> at all; then you'd have to reshape it into a form that it would be acceptable
> to the Linux kernel developers, who haven't seemed real keen on the idea so
> far.
given that RAID, snapshots, etc are already in the linux kernel, I suspect
that what will need to happen is for the filesystem to be ported without
those features and then the userspace tools (that manipulate the volumes )
be ported to use the things already in the kernel.
> The standard article I'm you've already seen this week on this topic is Jeff
> Bonwick's at http://blogs.sun.com/bonwick/entry/rampant_layering_violation
yep, that sounds like what I've been hearing.
what the ZFS (and reiserfs4) folks haven't been wanting to hear from the
linux kernel devs is that they are interested in having all these neat
features available for use with all filesystems (and the linux kernel has
a _lot_ of filesystems available), with solaris you basicly have UFS and
ZFS so it's not as big a deal.
> What really bugged me was his earlier article linked to there where he talks
> about how ZFS eliminates the need for hardware RAID controllers:
> http://blogs.sun.com/bonwick/entry/raid_z
>
> While there may be merit to that idea for some applications, like situations
> where you have a pig of a RAID5 volume, that's just hype for database writes.
> "We issue the SYNCHRONIZE CACHE command to the disks after pushing all data
> in a transaction group"--see, that would be the part the hardware controller
> is needed to accelerate. If you really care about whether your data hit
> disk, there is no way to break the RPM barrier without hardware support. The
> fact that he misunderstands such a fundamental point makes me wonder what
> other gigantic mistakes might be buried in his analysis.
I've seen similar comments from some of the linux kernel devs, they've
used low-end raid controllers with small processors on them and think that
a second core/socket in the main system to run software raid on is better.
David Lang
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 12:01 ` Re: Best OS for Postgres 8.2 Luke Lonergan <LLonergan@greenplum.com>
2007-05-09 03:51 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
2007-05-09 08:57 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-09 09:38 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2007-05-09 09:44 ` Re: Best OS for Postgres 8.2 david@lang.hm
0 siblings, 1 reply; 38+ messages in thread
From: Steinar H. Gunderson @ 2007-05-09 09:38 UTC (permalink / raw)
To: pgsql-performance
On Wed, May 09, 2007 at 01:57:51AM -0700, david@lang.hm wrote:
> given that RAID, snapshots, etc are already in the linux kernel, I suspect
> that what will need to happen is for the filesystem to be ported without
> those features and then the userspace tools (that manipulate the volumes )
> be ported to use the things already in the kernel.
Well, part of the idea behind ZFS is that these parts are _not_ separated in
"layers" -- for instance, the filesystem can push data down to the RAID level
to determine the stripe size used.
Whether this is a good idea is of course hotly debated, but I don't think you
can port just the filesystem part and call it a day.
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:49 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 10:29 ` Re: Best OS for Postgres 8.2 Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 12:01 ` Re: Best OS for Postgres 8.2 Luke Lonergan <LLonergan@greenplum.com>
2007-05-09 03:51 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
2007-05-09 08:57 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-09 09:38 ` Re: Best OS for Postgres 8.2 Steinar H. Gunderson <sgunderson@bigfoot.com>
@ 2007-05-09 09:44 ` david@lang.hm
0 siblings, 0 replies; 38+ messages in thread
From: david@lang.hm @ 2007-05-09 09:44 UTC (permalink / raw)
To: Steinar H. Gunderson <sgunderson@bigfoot.com>; +Cc: pgsql-performance
On Wed, 9 May 2007, Steinar H. Gunderson wrote:
> On Wed, May 09, 2007 at 01:57:51AM -0700, david@lang.hm wrote:
>> given that RAID, snapshots, etc are already in the linux kernel, I suspect
>> that what will need to happen is for the filesystem to be ported without
>> those features and then the userspace tools (that manipulate the volumes )
>> be ported to use the things already in the kernel.
>
> Well, part of the idea behind ZFS is that these parts are _not_ separated in
> "layers" -- for instance, the filesystem can push data down to the RAID level
> to determine the stripe size used.
there's nothing preventing this from happening if they are seperate layers
either.
there are some performance implications of the seperate layers, but until
someone has the ability to do head-to-head comparisons it's hard to say
which approach will win (in theory the lack of layers makes for faster
code, but in practice the fact that each layer is gone over by experts
looking for ways to optimize it may overwelm the layering overhead)
> Whether this is a good idea is of course hotly debated, but I don't think you
> can port just the filesystem part and call it a day.
Oh, I'm absolutly sure that doing so won't satidfy people (wnd would
generate howles of outrage from some parts), but having watched other
groups try and get things into the kernel that the kernel devs felt were
layering violations I think that it's wat will ultimatly happen.
David Lang
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
2007-05-08 08:22 ` Re: Best OS for Postgres 8.2 Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-11 13:35 ` Adam Witney <awitney@sgul.ac.uk>
1 sibling, 0 replies; 38+ messages in thread
From: Adam Witney @ 2007-05-11 13:35 UTC (permalink / raw)
To: pgsql-performance
> currently ZFS is only available on Solaris, parts of it have been released
> under GPLv2, but it doesn't look like enough of it to be ported to Linux
> (enough was released for grub to be able to access it read-only, but not
> the full filesystem). there are also patent concerns that are preventing
> any porting to Linux.
I don't know if anyone mentioned this in the thread already, but it looks
like ZFS may be coming to MacOSX 10.5
http://news.worldofapple.com/archives/2006/12/17/zfs-file-system-makes-it-to
-mac-os-x-leopard/
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 09:43 ` Alexander Staubo <alex@purefiction.net>
5 siblings, 0 replies; 38+ messages in thread
From: Alexander Staubo @ 2007-05-08 09:43 UTC (permalink / raw)
To: david@lang.hm <david@lang.hm>; +Cc: 李彦 Ian Li <liyan82@gmail.com>; pgsql-performance
On 5/8/07, david@lang.hm <david@lang.hm> wrote:
[snip]
> I personally don't trust reiserfs, jfs seems to be a tools for
> transitioning from AIX more then anything else [...]
What makes you say this? I have run JFS for years with complete
satisfaction, and I have never logged into an AIX box.
JFS has traditionally been seen as an underdog, but undeservedly so,
in my opinion; one cause might be the instability of the very early
releases, which seems to have tainted its reputation, or the alienness
of its AIX heritage. However, every benchmark I have come across puts
its on par with, and often surpassing, the more popular file systems
in performance. In particular, JFS seems to shine with respect to CPU
overhead.
Alexander.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 16:58 ` 李彦 Ian Li <liyan82@gmail.com>
5 siblings, 0 replies; 38+ messages in thread
From: 李彦 Ian Li @ 2007-05-08 16:58 UTC (permalink / raw)
To: ; +Cc: pgsql-performance
I am back with the chatlog and seem it's the Transparent compression
that helps a lot, very interesting...
here is the log of #postgresql on Apr. 21th around 13:20 GMT (snipped) :
<Solatis> why is that, when hard disk i/o is my bottleneck ?
<Solatis> well i have 10 disks in a raid1+0 config
<Solatis> it's sata2 yes
<Solatis> i run solaris express, whose kernel says SunOS
<Solatis> running 'SunOS solatis2 5.11 snv_61 i86pc i386 i86pc
<Solatis> well, the thing is, i'm using zfs
<Solatis> yeah, it was the reason for me to install solaris in
the first place
<Solatis> and a benchmark for my system comparing debian linux
with solaris express showed a +- 18% performance gain when switching
to solaris
<Solatis> so i'm happy
<Solatis> (note: the benchmarking was not scientifically
grounded at all, it was just around 50 million stored procedure
calls which do select/update/inserts on my database which would
simulate my specific case)
<Solatis> but the killer thing was to enable compression on zfs
<Solatis> that reduced the hard disk i/o with a factor 3, which
was the probable cause of the performance increase
<Solatis> oh, at the moment it's factor 2.23
<Solatis> still, it's funny to see that postgresql says that my
database is using around 41GB's, while only taking up 18GB on the
hard disk
=== end of log ===
david@lang.hm wrote:
> On Tue, 8 May 2007, �~]~N彦 Ian Li wrote:
>
>> In #postgresql on freenode, somebody ever mentioned that ZFS from
>> Solaris helps a lot to the performance of pgsql, so dose anyone have
>> information about that?
>
> the filesystem you use will affect the performance of postgres
> significantly. I've heard a lot of claims for ZFS, unfortunantly many of
> them from people who have prooven that they didn't know what they were
> talking about by the end of their first or second e-mails.
>
> much of the hype for ZFS is it's volume management capabilities and
> admin tools. Linux has most (if not all) of the volume management
> capabilities, it just seperates them from the filesystems so that any
> filesystem can use them, and as a result you use one tool to setup your
> RAID, one to setup snapshots, and a third to format your filesystems
> where ZFS does this in one userspace tool.
>
> once you seperate the volume management piece out, the actual
> performance question is a lot harder to answer. there are a lot of
> people who say that it's far faster then the alternate filesystems on
> Solaris, but I haven't seen any good comparisons between it and Linux
> filesystems.
>
> On Linux you have the choice of several filesystems, and the perfomance
> will vary wildly depending on your workload. I personally tend to favor
> ext2 (for small filesystems where the application is ensuring data
> integrity) or XFS (for large filesystems)
>
> I personally don't trust reiserfs, jfs seems to be a tools for
> transitioning from AIX more then anything else, and ext3 seems to have
> all the scaling issues of ext2 plus the overhead (and bottleneck) of
> journaling.
>
> one issue with journaling filesystems, if you journal the data as well
> as the metadata you end up with a very reliable setup, however it means
> that all your data needs to be written twice, oncce to the journal, and
> once to the final location. the write to the journal can be slightly
> faster then a normal write to the final location (the journal is a
> sequential write to an existing file), however the need to write twice
> can effectivly cut your disk I/O bandwidth in half when doing heavy
> writes. worse, when you end up writing mor ethen will fit in the journal
> (128M is the max for ext3) the entire system then needs to stall while
> the journal gets cleared to make space for the additional writes.
>
> if you don't journal your data then you avoid the problems above, but in
> a crash you may find that you lost data, even though the filesystem is
> 'intact' according to fsck.
>
> David Lang
>
Regards
Ian
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 20:49 ` Charles Sprickman <spork@bway.net>
5 siblings, 0 replies; 38+ messages in thread
From: Charles Sprickman @ 2007-05-08 20:49 UTC (permalink / raw)
To: david@lang.hm; +Cc: 李彦 Ian Li <liyan82@gmail.com>; pgsql-performance
On Tue, 8 May 2007, david@lang.hm wrote:
> one issue with journaling filesystems, if you journal the data as well as the
> metadata you end up with a very reliable setup, however it means that all
> your data needs to be written twice, oncce to the journal, and once to the
> final location. the write to the journal can be slightly faster then a normal
> write to the final location (the journal is a sequential write to an existing
> file), however the need to write twice can effectivly cut your disk I/O
> bandwidth in half when doing heavy writes. worse, when you end up writing mor
> ethen will fit in the journal (128M is the max for ext3) the entire system
> then needs to stall while the journal gets cleared to make space for the
> additional writes.
>
> if you don't journal your data then you avoid the problems above, but in a
> crash you may find that you lost data, even though the filesystem is 'intact'
> according to fsck.
That sounds like an ad for FreeBSD and UFS2+Softupdates. :)
Metadata is as safe as it is in a journaling filesystem, but none of the
overhead of journaling.
Charles
> David Lang
>
>> Steve Atkins wrote:
>>>
>>> On May 7, 2007, at 2:55 PM, David Levy wrote:
>>>
>>> > Hi,
>>> > > I am about to order a new server for my Postgres cluster. I will
>>> > probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>>> > Which OS would you recommend to optimize Postgres behaviour (i/o
>>> > access, multithreading, etc) ?
>>> > > I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>>> > help with this ?
>>>
>>> Well, all three you mention are much the same, just with a different
>>> badge on the box, as far as performance is concerned. They're all
>>> going to be a moderately recent Linux kernel, with your choice
>>> of filesystems, so any choice between them is going to be driven
>>> more by available staff and support or personal preference.
>>>
>>> I'd probably go CentOS 5 over Fedora just because Fedora doesn't
>>> get supported for very long - more of an issue with a dedicated
>>> database box with a long lifespan than your typical desktop or
>>> interchangeable webserver.
>>>
>>> I might also look at Solaris 10, though. I've yet to play with it much,
>>> but it
>>> seems nice, and I suspect it might manage 8 cores better than current
>>> Linux setups.
>>>
>>> Cheers,
>>> Steve
>>>
>>>
>>>
>>> ---------------------------(end of broadcast)---------------------------
>>> TIP 5: don't forget to increase your free space map settings
>>>
>>
>> Regards
>>
>> Ian
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 3: Have you checked our extensive FAQ?
>>
>> http://www.postgresql.org/docs/faq
>>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 22:57 ` Re: Best OS for Postgres 8.2 Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` Re: Best OS for Postgres 8.2 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-09 17:10 ` Jim Nasby <decibel@decibel.org>
5 siblings, 0 replies; 38+ messages in thread
From: Jim Nasby @ 2007-05-09 17:10 UTC (permalink / raw)
To: david@lang.hm; +Cc: 李彦 Ian Li <liyan82@gmail.com>; pgsql-performance
On May 8, 2007, at 2:59 AM, david@lang.hm wrote:
> one issue with journaling filesystems, if you journal the data as
> well as the metadata you end up with a very reliable setup, however
> it means that all your data needs to be written twice, oncce to the
> journal, and once to the final location. the write to the journal
> can be slightly faster then a normal write to the final location
> (the journal is a sequential write to an existing file), however
> the need to write twice can effectivly cut your disk I/O bandwidth
> in half when doing heavy writes. worse, when you end up writing mor
> ethen will fit in the journal (128M is the max for ext3) the entire
> system then needs to stall while the journal gets cleared to make
> space for the additional writes.
That's why you want to mount ext3 partitions used with PostgreSQL
with data=writeback.
Some folks will also use a small filesystem for pg_xlog and mount
that as ext2.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
@ 2007-05-08 00:40 ` Chris <dmagick@gmail.com>
2007-05-08 00:53 ` Re: Best OS for Postgres 8.2 david@lang.hm
4 siblings, 1 reply; 38+ messages in thread
From: Chris @ 2007-05-08 00:40 UTC (permalink / raw)
To: David Levy <dvid.levy@gmail.com>; +Cc: pgsql-performance
David Levy wrote:
> Hi,
>
> I am about to order a new server for my Postgres cluster. I will
> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
> Which OS would you recommend to optimize Postgres behaviour (i/o
> access, multithreading, etc) ?
>
> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
> help with this ?
Use the one you're most comfortable with.
I don't think you'll notice *that* much difference between linux systems
for performance - but whether you're comfortable using any of them will
make a difference in managing it in general.
--
Postgresql & php tutorials
http://www.designmagick.com/
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 00:40 ` Re: Best OS for Postgres 8.2 Chris <dmagick@gmail.com>
@ 2007-05-08 00:53 ` david@lang.hm
2007-05-08 01:14 ` Re: Best OS for Postgres 8.2 Chris <dmagick@gmail.com>
0 siblings, 1 reply; 38+ messages in thread
From: david@lang.hm @ 2007-05-08 00:53 UTC (permalink / raw)
To: Chris <dmagick@gmail.com>; +Cc: David Levy <dvid.levy@gmail.com>; pgsql-performance
On Tue, 8 May 2007, Chris wrote:
> David Levy wrote:
>> Hi,
>>
>> I am about to order a new server for my Postgres cluster. I will
>> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>> Which OS would you recommend to optimize Postgres behaviour (i/o
>> access, multithreading, etc) ?
>>
>> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>> help with this ?
>
> Use the one you're most comfortable with.
>
> I don't think you'll notice *that* much difference between linux systems for
> performance - but whether you're comfortable using any of them will make a
> difference in managing it in general.
the tuneing that you do (both of the OS and of postgres) will make more
of a difference then anything else.
David Lang
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 00:40 ` Re: Best OS for Postgres 8.2 Chris <dmagick@gmail.com>
2007-05-08 00:53 ` Re: Best OS for Postgres 8.2 david@lang.hm
@ 2007-05-08 01:14 ` Chris <dmagick@gmail.com>
0 siblings, 0 replies; 38+ messages in thread
From: Chris @ 2007-05-08 01:14 UTC (permalink / raw)
To: david@lang.hm; +Cc: David Levy <dvid.levy@gmail.com>; pgsql-performance
david@lang.hm wrote:
> On Tue, 8 May 2007, Chris wrote:
>
>> David Levy wrote:
>>> Hi,
>>>
>>> I am about to order a new server for my Postgres cluster. I will
>>> probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>>> Which OS would you recommend to optimize Postgres behaviour (i/o
>>> access, multithreading, etc) ?
>>>
>>> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>>> help with this ?
>>
>> Use the one you're most comfortable with.
>>
>> I don't think you'll notice *that* much difference between linux
>> systems for performance - but whether you're comfortable using any of
>> them will make a difference in managing it in general.
>
> the tuneing that you do (both of the OS and of postgres) will make more
> of a difference then anything else.
Which is why it's best to know/understand the OS first ;)
--
Postgresql & php tutorials
http://www.designmagick.com/
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
@ 2007-05-08 03:41 ` Ron <rjpeace@earthlink.net>
4 siblings, 0 replies; 38+ messages in thread
From: Ron @ 2007-05-08 03:41 UTC (permalink / raw)
To: David Levy <dvid.levy@gmail.com>; pgsql-performance
I am using FC6 in production for our pg 8.2.4 DB server and am quite
happy with it.
The big advantage with FC6 for me was that the FC6 team seems to keep
more current with the latest stable revs of most OSSW (including
kernel revs!) better than any of the other major distros.
(Also, SE Linux is a =good= thing security-wise. If it's good enough
for the NSA...)
Downside is that initial install and config can be a bit complicated.
We're happy with it.
Cheers,
Ron Peacetree
At 05:55 PM 5/7/2007, David Levy wrote:
>Hi,
>
>I am about to order a new server for my Postgres cluster. I will
>probably get a Dual Xeon Quad Core instead of my current Dual Xeon.
>Which OS would you recommend to optimize Postgres behaviour (i/o
>access, multithreading, etc) ?
>
>I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
>help with this ?
>
>
>Regards
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
@ 2007-05-08 03:56 ` Greg Smith <gsmith@gregsmith.com>
2007-05-08 04:37 ` Re: Best OS for Postgres 8.2 Tom Lane <tgl@sss.pgh.pa.us>
2007-05-08 09:35 ` Re: Best OS for Postgres 8.2 Steinar H. Gunderson <sgunderson@bigfoot.com>
4 siblings, 2 replies; 38+ messages in thread
From: Greg Smith @ 2007-05-08 03:56 UTC (permalink / raw)
To: pgsql-performance
On Mon, 7 May 2007, David Levy wrote:
> I am hesitating between Fedora Core 6, CentOS and Debian. Can anyone
> help with this ?
Debian packages PostgreSQL in a fashion unique to it; it's arguable
whether it's better or not (I don't like it), but going with that will
assure your installation is a bit non-standard compared with most Linux
installas. The main reasons you'd pick Debian are either that you like
that scheme (which tries to provide some structure to running multiple
clusters on one box), or that you plan to rely heavily on community
packages that don't come with the Redhat distributions and therefore would
appreciate how easy it is to use apt-get against the large Debian software
repository.
Given the buginess and unexpected changes from packages updates of every
Fedora Core release I've ever tried, I wouldn't trust any OS from that
line to run a database keeping track of where my socks are at. Core 6
seems better than most of the older ones. I find it hard to understand
what it offers that Centos doesn't such that you'd want Fedora instead.
Centos just released a new version 5 recently. It's running a fairly
modern kernel with several relevant performance improvements over the much
older V4; unless you have some odd piece of hardware where there is only a
driver available for Centos 4 (I ran into this with a disk controller),
the new version would better.
The main advantages of Centos over the other two are that so many people
are/will be running very similar configurations that you should able to
find help easily if you run into any issues. I revisited fresh installs
of each recently, and after trying both I found it more comfortable to run
the database server on Centos, but I did miss the gigantic and easy to
install Debian software repository.
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 03:56 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
@ 2007-05-08 04:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2007-05-09 03:31 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
1 sibling, 1 reply; 38+ messages in thread
From: Tom Lane @ 2007-05-08 04:37 UTC (permalink / raw)
To: Greg Smith <gsmith@gregsmith.com>; +Cc: pgsql-performance
Greg Smith <gsmith@gregsmith.com> writes:
> Debian packages PostgreSQL in a fashion unique to it; it's arguable
> whether it's better or not (I don't like it), but going with that will
> assure your installation is a bit non-standard compared with most Linux
> installas.
<dons red fedora>
What Debian has done is set up an arrangement that lets you run two (or
more) different PG versions in parallel. Since that's amazingly helpful
during a major-PG-version upgrade, most of the other packagers are
scheming how to do something similar. I'm not sure when this will
happen in the PGDG or Red Hat RPMs, but it probably will eventually.
> Given the buginess and unexpected changes from packages updates of every
> Fedora Core release I've ever tried, I wouldn't trust any OS from that
> line to run a database keeping track of where my socks are at. Core 6
> seems better than most of the older ones. I find it hard to understand
> what it offers that Centos doesn't such that you'd want Fedora instead.
Fedora is about cutting edge, RHEL is about stability, and Centos tracks
RHEL. No surprises there. (<plug> and if someday you want commercial
support for your OS, a Centos->RHEL update will get you there easily.
AFAIK Red Hat doesn't have a clean solution for someone running Fedora
who suddenly realizes he needs a 24x7-supportable OS right now.
Something to work on... </plug>)
</dons red fedora>
regards, tom lane
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 03:56 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
2007-05-08 04:37 ` Re: Best OS for Postgres 8.2 Tom Lane <tgl@sss.pgh.pa.us>
@ 2007-05-09 03:31 ` Greg Smith <gsmith@gregsmith.com>
2007-05-12 03:12 ` Re: Best OS for Postgres 8.2 Robert Treat <xzilla@users.sourceforge.net>
2007-05-13 06:34 ` Re: Best OS for Postgres 8.2 Andrew McMillan <andrew@catalyst.net.nz>
0 siblings, 2 replies; 38+ messages in thread
From: Greg Smith @ 2007-05-09 03:31 UTC (permalink / raw)
To: pgsql-performance
On Tue, 8 May 2007, Tom Lane wrote:
> What Debian has done is set up an arrangement that lets you run two (or
> more) different PG versions in parallel. Since that's amazingly helpful
> during a major-PG-version upgrade, most of the other packagers are
> scheming how to do something similar.
I alluded to that but it is worth going into more detail on for those not
familiar with this whole topic. I normally maintain multiple different PG
versions in parallel already, mostly using environment variables to switch
between them with some shell code. Debian has taken an approach where
commands like pg_ctl are wrapped in multi-version/cluster aware scripts,
so you can do things like restarting multiple installations more easily
than that.
My issue wasn't with the idea, it was with the implementation. When I
have my newbie hat on, it adds a layer of complexity that isn't needed for
simple installs. And when I have my developer hat on, I found that need
to conform to the requirements of that system on top of Debian's already
unique install locations and packaging issues just made it painful to
build and work with with customized versions of Postgres, compared to
distributions that use a relatively simple packaging scheme (like the RPM
based RedHat or SuSE).
I hope anyone else working this problem is thinking about issues like
this. Debian's approach strikes me as being a good one for a seasoned
systems administrator or DBA, which is typical for them. I'd hate to see
a change in this area make it more difficult for new users though, as
that's already perceived as a PG weakness. I think you can build a layer
that adds the capability for the people who need it without complicating
things for people who don't.
> and if someday you want commercial support for your OS, a Centos->RHEL
> update will get you there easily.
For those that like to live dangerously, it's also worth mentioning that
it's possible to hack this conversion in either direction without actually
doing an OS re-install/upgrade just by playing with the packages that are
different between the two. So someone who installs CentOS now could swap
to RHEL very quickly in a pinch if they have enough cojones to do the
required package substitutions.
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 03:56 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
2007-05-08 04:37 ` Re: Best OS for Postgres 8.2 Tom Lane <tgl@sss.pgh.pa.us>
2007-05-09 03:31 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
@ 2007-05-12 03:12 ` Robert Treat <xzilla@users.sourceforge.net>
1 sibling, 0 replies; 38+ messages in thread
From: Robert Treat @ 2007-05-12 03:12 UTC (permalink / raw)
To: pgsql-performance; +Cc: Greg Smith <gsmith@gregsmith.com>
On Tuesday 08 May 2007 23:31, Greg Smith wrote:
> On Tue, 8 May 2007, Tom Lane wrote:
> > What Debian has done is set up an arrangement that lets you run two (or
> > more) different PG versions in parallel. Since that's amazingly helpful
> > during a major-PG-version upgrade, most of the other packagers are
> > scheming how to do something similar.
>
> I alluded to that but it is worth going into more detail on for those not
> familiar with this whole topic. I normally maintain multiple different PG
> versions in parallel already, mostly using environment variables to switch
> between them with some shell code. Debian has taken an approach where
> commands like pg_ctl are wrapped in multi-version/cluster aware scripts,
> so you can do things like restarting multiple installations more easily
> than that.
>
> My issue wasn't with the idea, it was with the implementation. When I
> have my newbie hat on, it adds a layer of complexity that isn't needed for
> simple installs.
I think I would disagree with this. The confusion comes from the fact that it
is different, not that it is more complex. For new users what seems to be
most confusing is getting from install to initdb to logging in... if you tell
them to use pg_ctlcluster rather than pg_ctl, it isn't more confusing, there
just following directions at that point anyway. If the upstream project were
to switch to debian's system, I think you'd end most of the confusion, make
it easier to run concurrent servers and simplify the upgrade process for
source installs, and give other package maintiners a way to achive what
debian has. Maybe in PG 9...
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 03:56 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
2007-05-08 04:37 ` Re: Best OS for Postgres 8.2 Tom Lane <tgl@sss.pgh.pa.us>
2007-05-09 03:31 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
@ 2007-05-13 06:34 ` Andrew McMillan <andrew@catalyst.net.nz>
1 sibling, 0 replies; 38+ messages in thread
From: Andrew McMillan @ 2007-05-13 06:34 UTC (permalink / raw)
To: Greg Smith <gsmith@gregsmith.com>; +Cc: pgsql-performance
On Tue, 2007-05-08 at 23:31 -0400, Greg Smith wrote:
>
> My issue wasn't with the idea, it was with the implementation. When I
> have my newbie hat on, it adds a layer of complexity that isn't needed for
> simple installs.
I find it very hard to agree with that.
As a newbie I install postgresql and have a database server installed,
and operating. The fact that the DB files are installed somewhere
like /var/lib/postgresql/8.1/main is waaay beyond newbie.
At that point I can "createdb" or "createuser", but I _do_ _not_ need to
know anything about the cluster stuff until there is more than one DB on
the machine.
The Debian wrappers all default appropriately for the single-cluster
case, so having a single cluster has added _no_ perceivable complexity
for a newbie (as it should).
If you have a second cluster, whether it's the same Pg version or not,
things necessarily start to get complicated. OTOH I haven't had any
problem explaining to people that the --cluster option applies, and
there are sane ways to make that default to a reasonable thing as well.
All in all I think that the Debian scripts are excellent. I'm sure
there are improvements that could be made, but overall they don't get in
the way, they do the right thing in the minimal case, and they give the
advanced user a lot more choices about multiple DB instances on the same
machine.
Cheers,
Andrew McMillan.
-------------------------------------------------------------------------
Andrew @ Catalyst .Net .NZ Ltd, PO Box 11-053, Manners St, Wellington
WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St
DDI: +64(4)803-2201 MOB: +64(272)DEBIAN OFFICE: +64(4)499-2267
Open Source: the difference between trust and antitrust
-------------------------------------------------------------------------
Attachments:
signature.asc (application/pgp-signature, 188B) download
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Best OS for Postgres 8.2
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-08 03:56 ` Re: Best OS for Postgres 8.2 Greg Smith <gsmith@gregsmith.com>
@ 2007-05-08 09:35 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
1 sibling, 0 replies; 38+ messages in thread
From: Steinar H. Gunderson @ 2007-05-08 09:35 UTC (permalink / raw)
To: pgsql-performance
On Mon, May 07, 2007 at 11:56:14PM -0400, Greg Smith wrote:
> Debian packages PostgreSQL in a fashion unique to it; it's arguable
> whether it's better or not (I don't like it), but going with that will
> assure your installation is a bit non-standard compared with most Linux
> installas. The main reasons you'd pick Debian are either that you like
> that scheme (which tries to provide some structure to running multiple
> clusters on one box), or that you plan to rely heavily on community
> packages that don't come with the Redhat distributions and therefore would
> appreciate how easy it is to use apt-get against the large Debian software
> repository.
Just to add to this: As far as I understand it, this scheme was originally
mainly put in place to allow multiple _versions_ of Postgres to be installed
alongside each other, for smoother upgrades. (There's a command that does all
the details of running first pg_dumpall for the users and groups, then the
new pg_dump with -Fc to get all data and LOBs over, then some hand-fixing to
change explicit paths to $libdir, etc...)
Of course, you lose all that if you need a newer Postgres version than the OS
provides. (Martin Pitt, the Debian/Ubuntu maintainer of Postgres -- the
packaging in Debian and Ubuntu is the same, sans version differences -- makes
his own backported packages of the newest Postgres to Debian stable; it's up
to you if you'd trust that or not.)
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 38+ messages in thread
end of thread, other threads:[~2007-05-13 06:34 UTC | newest]
Thread overview: 38+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-05-07 21:55 Best OS for Postgres 8.2 David Levy <dvid.levy@gmail.com>
2007-05-07 21:59 ` Joshua D. Drake <jd@commandprompt.com>
2007-05-07 22:04 ` Bill Moran <wmoran@collaborativefusion.com>
2007-05-07 22:14 ` Joshua D. Drake <jd@commandprompt.com>
2007-05-08 09:48 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2007-05-08 10:27 ` david@lang.hm
2007-05-07 22:57 ` Steve Atkins <steve@blighty.com>
2007-05-08 04:41 ` 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 07:59 ` david@lang.hm
2007-05-08 08:21 ` Heikki Linnakangas <heikki@enterprisedb.com>
2007-05-08 08:22 ` Claus Guttesen <kometen@gmail.com>
2007-05-08 08:45 ` david@lang.hm
2007-05-08 08:49 ` Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:16 ` david@lang.hm
2007-05-08 10:29 ` Trygve Laugstøl <trygvis@inamo.no>
2007-05-08 10:55 ` "C. Bergström" <cbergstrom@netsyncro.com>
2007-05-08 12:25 ` Adam Tauno Williams <adamtaunowilliams@gmail.com>
2007-05-08 12:01 ` Luke Lonergan <LLonergan@greenplum.com>
2007-05-08 16:08 ` Ron <rjpeace@earthlink.net>
2007-05-09 03:51 ` Greg Smith <gsmith@gregsmith.com>
2007-05-09 08:57 ` david@lang.hm
2007-05-09 09:38 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2007-05-09 09:44 ` david@lang.hm
2007-05-11 13:35 ` Adam Witney <awitney@sgul.ac.uk>
2007-05-08 09:43 ` Alexander Staubo <alex@purefiction.net>
2007-05-08 16:58 ` 李彦 Ian Li <liyan82@gmail.com>
2007-05-08 20:49 ` Charles Sprickman <spork@bway.net>
2007-05-09 17:10 ` Jim Nasby <decibel@decibel.org>
2007-05-08 00:40 ` Chris <dmagick@gmail.com>
2007-05-08 00:53 ` david@lang.hm
2007-05-08 01:14 ` Chris <dmagick@gmail.com>
2007-05-08 03:41 ` Ron <rjpeace@earthlink.net>
2007-05-08 03:56 ` Greg Smith <gsmith@gregsmith.com>
2007-05-08 04:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2007-05-09 03:31 ` Greg Smith <gsmith@gregsmith.com>
2007-05-12 03:12 ` Robert Treat <xzilla@users.sourceforge.net>
2007-05-13 06:34 ` Andrew McMillan <andrew@catalyst.net.nz>
2007-05-08 09:35 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox