From: Andres Freund <andres@anarazel.de>
To: pgsql-hackers@postgresql.org
Cc: Mark Mielke <mark@mark.mielke.cc>
Cc: Florian Weimer <fw@deneb.enyo.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Greg Stark <gsstark@mit.edu>
Cc: Robert Haas <robertmhaas@gmail.com>
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync
Date: Sun, 14 Feb 2010 21:49:09 +0100
Message-ID: <201002142149.12786.andres@anarazel.de> (raw)
In-Reply-To: <4B785FDE.8040308@mark.mielke.cc>
References: <4427a97a0912101235x3ade510blcc00d5ce9883c6b6@mail.gmail.com>
<87wryfpnuv.fsf@mid.deneb.enyo.de>
<4B785FDE.8040308@mark.mielke.cc>
On Sunday 14 February 2010 21:41:02 Mark Mielke wrote:
> On 02/14/2010 03:24 PM, Florian Weimer wrote:
> > * Tom Lane:
> >>> Which options would that be? I am not aware that there any for any of
> >>> the recent linux filesystems.
> >>
> >> Shouldn't journaling of metadata be sufficient?
> >
> > You also need to enforce ordering between the directory update and the
> > file update. The file metadata is flushed with fsync(), but the
> > directory isn't. On some systems, all directory operations are
> > synchronous, but not on Linux.
>
> dirsync
> All directory updates within the filesystem should be
> done syn-
> chronously. This affects the following system calls:
> creat,
> link, unlink, symlink, mkdir, rmdir, mknod and rename.
>
> The widely reported problems, though, did not tend to be a problem with
> directory changes written too late - but directory changes being written
> too early. That is, the directory change is written to disk, but the
> file content is not. This is likely because of the "ordered journal"
> mode widely used in ext3/ext4 where metadata changes are journalled, but
> file pages are not journalled. Therefore, it is important for some
> operations, that the file pages are pushed to disk using fsync(file),
> before the metadata changes are journalled.
Well, but thats not a problem with pg as it fsyncs the file contents.
> In theory there is some open hole where directory updates need to be
> synchronized with file updates, as POSIX doesn't enforce this ordering,
> and we can't trust that all file systems implicitly order things
> correctly, but in practice, I don't see this sort of problem happening.
I can try to reproduce it if you want...
> If you are concerned, enable dirsync.
If the filesystem already behaves that way a fsync on it should be fairly
cheap. If it doesnt behave that way doing it is correct...
Besides there is no reason to fsync the directory before the checkpoint, so
dirsync would require a higher cost than doing it correctly.
Andres
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: andres@anarazel.de, pgsql-hackers@postgresql.org, mark@mark.mielke.cc, fw@deneb.enyo.de, tgl@sss.pgh.pa.us, gsstark@mit.edu, robertmhaas@gmail.com
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync
In-Reply-To: <201002142149.12786.andres@anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox