From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu
	karmic slow createdb)
Date: Sun, 14 Feb 2010 22:43:23 +0100
User-Agent: KMail/1.13.0 (Linux/2.6.33-rc7-andres-00124-g828206f; KDE/4.4.0;
	x86_64; ; )
Cc: Greg Stark <gsstark@mit.edu>,
 pgsql-hackers@postgresql.org
References: <4427a97a0912101235x3ade510blcc00d5ce9883c6b6@mail.gmail.com>
	<407d949e1002140731j85a4f97nc053ec1b0f3cc458@mail.gmail.com>
	<603c8f071002141257g2dfb7a09i5c5930e9787783e1@mail.gmail.com>
In-Reply-To: <603c8f071002141257g2dfb7a09i5c5930e9787783e1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Message-Id: <201002142243.24871.andres@anarazel.de>

On Sunday 14 February 2010 21:57:08 Robert Haas wrote:
> On Sun, Feb 14, 2010 at 10:31 AM, Greg Stark <gsstark@mit.edu> wrote:
> > On Sun, Feb 14, 2010 at 2:03 PM, Greg Stark <gsstark@mit.edu> wrote:
> >> On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas <robertmhaas@gmail.com> 
wrote:
> >>> Greg Stark, have you managed to get your access issues sorted out?  If
> >> 
> >> Yep, will look at this today.
> > 
> > So I think we have a bigger problem than just copydir.c. It seems to
> > me we should be fsyncing the table space data directories on every
> > checkpoint. Otherwise any newly created relations or removed relations
> > could disappear even though the data in them was fsynced. I'm thinking
> > I should add an _mdfd_opentblspc(reln) call which returns a file
> > descriptor for the tablespace and have mdsync() use that to sync the
> > directory whenever it fsyncs a relation. It would be nice to remember
> > which tablespaces have been fsynced and only fsync them once though,
> > that would need another hash table just for tablespaces.
> > 
> > We probably also need to fsync the pg_xlog directory every time we
> > create or rename an xlog segment.
> > 
> > Are there any other places we do directory operations which we need to
> > be permanent?
> 
> I agree with Tom that we need to see some actual reproducible test
> cases where this is an issue before we go too crazy with it.  In
> theory what you're talking about could also happen when extending a
> relation, if we extend into a new file; but I think we need to
> convince ourselves that it really happens before we make any more
> changes.
Ok, will try to reproduce.

> On a pragmatic note, if this does turn out to be a problem, it's a
> bug: and we can and do fix bugs whenever we discover them.  But the
> other part of this patch - to speed up createdb - is a feature - and
> we are very rapidly running out of time for 9.0 features.  So I'd like
> to vote for getting the feature part of this committed (assuming it's
> in good shape, of course) and we can continue to investigate the other
> issues but without quite as much urgency.
Sound sensible.

Andres