pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedLinux Filesystems again - Ubuntu this time
7+ messages / 3 participants
[nested] [flat]
* Linux Filesystems again - Ubuntu this time
@ 2010-07-27 05:04 Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
0 siblings, 1 reply; 7+ messages in thread
From: Mark Kirkwood @ 2010-07-27 05:04 UTC (permalink / raw)
To: pgsql-performance
I'm involved with the setup of replacement hardware for one of our
systems. It is going to be using Ubuntu Lucid Server (kernel 2.6.32 I
believe). The issue of filesystems has raised its head again.
I note that ext4 is now the default for Lucid, what do folks think about
using it: stable enough now? Also xfs has seen quite a bit of
development in these later kernels, any thoughts on that?
Cheers
Mark
P.s: We are quite keen to move away from ext3, as we have encountered
its tendency to hit a wall under heavy load and leave us waiting for
kjournald and pdflush to catch up....
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Linux Filesystems again - Ubuntu this time
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
@ 2010-07-27 13:48 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 16:57 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Kevin Grittner @ 2010-07-27 13:48 UTC (permalink / raw)
To: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>; pgsql-performance
Mark Kirkwood <mark.kirkwood@catalyst.net.nz> wrote:
> Also xfs has seen quite a bit of development in these later
> kernels, any thoughts on that?
We've been using xfs for a few years now with good performance and
no problems other than needing to disable write barriers to get good
performance out of our battery-backed RAID adapter.
-Kevin
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Linux Filesystems again - Ubuntu this time
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
@ 2010-07-27 16:57 ` Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 17:10 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
0 siblings, 1 reply; 7+ messages in thread
From: Whit Armstrong @ 2010-07-27 16:57 UTC (permalink / raw)
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>; +Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>; pgsql-performance
Kevin,
While we're on the topic, do you also diable fsync?
We use xfs with battery-backed raid as well. We have had no issues with xfs.
I'm curious whether anyone can comment on his experience (good or bad)
using xfs/battery-backed-cache/fsync=off.
Thanks,
Whit
On Tue, Jul 27, 2010 at 9:48 AM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:
> Mark Kirkwood <mark.kirkwood@catalyst.net.nz> wrote:
>
>> Also xfs has seen quite a bit of development in these later
>> kernels, any thoughts on that?
>
> We've been using xfs for a few years now with good performance and
> no problems other than needing to disable write barriers to get good
> performance out of our battery-backed RAID adapter.
>
> -Kevin
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Linux Filesystems again - Ubuntu this time
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 16:57 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
@ 2010-07-27 17:10 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 17:19 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
0 siblings, 1 reply; 7+ messages in thread
From: Kevin Grittner @ 2010-07-27 17:10 UTC (permalink / raw)
To: Whit Armstrong <armstrong.whit@gmail.com>; +Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>; pgsql-performance
Whit Armstrong <armstrong.whit@gmail.com> wrote:
> While we're on the topic, do you also diable fsync?
We only disable fsync during bulk loads, where we would be starting
over anyway if there was a failure. Basically, you should never use
fsync unless you are OK with losing everything in the database
server if you have an OS or hardware failure. We have a few
databases where we would consider that if performance wasn't
otherwise acceptable, since they are consolidated replicas of
off-side source databases, and we have four identical ones in two
separate buildings; however, since performance is good with fsync on
and it would be a bother to have to copy from one of the other
servers in the event of an OS crash, we leave it on.
-Kevin
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Linux Filesystems again - Ubuntu this time
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 16:57 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 17:10 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
@ 2010-07-27 17:19 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 18:20 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Kevin Grittner @ 2010-07-27 17:19 UTC (permalink / raw)
To: Whit Armstrong <armstrong.whit@gmail.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; +Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>; pgsql-performance
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> wrote:
> Basically, you should never use fsync unless you are OK with
> losing everything in the database server if you have an OS or
> hardware failure.
s/use/disable/
-Kevin
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Linux Filesystems again - Ubuntu this time
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 16:57 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 17:10 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 17:19 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
@ 2010-07-27 18:20 ` Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 18:32 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
0 siblings, 1 reply; 7+ messages in thread
From: Whit Armstrong @ 2010-07-27 18:20 UTC (permalink / raw)
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>; +Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>; pgsql-performance
Thanks.
But there is no such risk to turning off write barriers?
I'm only specifying noatime for xfs at the moment.
Did you get a substantial performace boost from disabling write
barriers? like 10x or more like 2x?
Thanks,
Whit
On Tue, Jul 27, 2010 at 1:19 PM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:
> "Kevin Grittner" <Kevin.Grittner@wicourts.gov> wrote:
>
>> Basically, you should never use fsync unless you are OK with
>> losing everything in the database server if you have an OS or
>> hardware failure.
>
> s/use/disable/
>
> -Kevin
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Linux Filesystems again - Ubuntu this time
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 16:57 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 17:10 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 17:19 ` Re: Linux Filesystems again - Ubuntu this time Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 18:20 ` Re: Linux Filesystems again - Ubuntu this time Whit Armstrong <armstrong.whit@gmail.com>
@ 2010-07-27 18:32 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
0 siblings, 0 replies; 7+ messages in thread
From: Kevin Grittner @ 2010-07-27 18:32 UTC (permalink / raw)
To: Whit Armstrong <armstrong.whit@gmail.com>; +Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>; pgsql-performance
Whit Armstrong <armstrong.whit@gmail.com> wrote:
> But there is no such risk to turning off write barriers?
Supposedly not:
http://xfs.org/index.php/XFS_FAQ#Q._Should_barriers_be_enabled_with_storage_which_has_a_persistent_w...
> Did you get a substantial performace boost from disabling write
> barriers? like 10x or more like 2x?
It made a huge difference on creation and deletion of disk files.
Unfortunately we have some procedures which use a cursor and loop
through rows calling a function which creates and drops a temporary
table. While I would like to see those transactions rewritten to
use sane techniques, they run fast enough without the write barriers
to be acceptable to the users, which puts the issue pretty low on
the priority list. I don't have the numbers anymore, but I'm sure
it was closer to 100 times slower than 10 times. In some workloads
you might not notice the difference, although I would watch out for
checkpoint behavior.
-Kevin
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2010-07-27 18:32 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-07-27 05:04 Linux Filesystems again - Ubuntu this time Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2010-07-27 13:48 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 16:57 ` Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 17:10 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 17:19 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-07-27 18:20 ` Whit Armstrong <armstrong.whit@gmail.com>
2010-07-27 18:32 ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox