From: Andres Freund <andres@anarazel.de>
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1
	ubuntu karmic slow createdb)
Date: Thu, 11 Feb 2010 03:27:30 +0100
User-Agent: KMail/1.13.0 (Linux/2.6.33-rc5-andres-00243-g6ea299e; KDE/4.4.0;
	x86_64; ; )
Cc: Robert Haas <robertmhaas@gmail.com>,
	Alvaro Herrera <alvherre@commandprompt.com>, Tom Lane <tgl@sss.pgh.pa.us>,
	Greg Smith <greg@2ndquadrant.com>, Greg Stark <gsstark@mit.edu>,
	pgsql-performance@postgresql.org,
	Michael Clemmons <glassresistor@gmail.com>,
	Hannu Krosing <hannu@2ndquadrant.com>,
	Scott Marlowe <scott.marlowe@gmail.com>
References: <4427a97a0912101235x3ade510blcc00d5ce9883c6b6@mail.gmail.com>
	<20100208030901.GA7407@alvh.no-ip.org>
	<603c8f071002072053w16897addtd52fcc4cf880df53@mail.gmail.com>
In-Reply-To: <603c8f071002072053w16897addtd52fcc4cf880df53@mail.gmail.com>
MIME-Version: 1.0
Content-Type: Multipart/Mixed;
  boundary="Boundary-00=_Ts2cLvEAv/Qj+ul"
Message-Id: <201002110327.35810.andres@anarazel.de>

--Boundary-00=_Ts2cLvEAv/Qj+ul
Content-Type: Text/Plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

On Monday 08 February 2010 05:53:23 Robert Haas wrote:
> On Sun, Feb 7, 2010 at 10:09 PM, Alvaro Herrera
>=20
> <alvherre@commandprompt.com> wrote:
> > Andres Freund escribi=F3:
> >> I personally think the fsync on the directory should be added to the
> >> stable branches - other opinions?
> >> If wanted I can prepare patches for that.
> >=20
> > Yeah, it seems there are two patches here -- one is the addition of
> > fsync_fname() and the other is the fsync_prepare stuff.
>=20
> Andres, you want to take a crack at splitting this up?
I hope I didnt duplicate Gregs work, but I didnt hear back from him, so...

Everything <8.1 is hopeless because cp is used there... I didnt see it wort=
h=20
to replace that. The patch applies cleanly for 8.1 to 8.4 and survives the=
=20
regression tests

Given pg's heavy commit model I didnt see a point to split the patch for 9.=
0=20
as well...

Andres

--Boundary-00=_Ts2cLvEAv/Qj+ul
Content-Type: text/x-patch; charset="ISO-8859-1";
	name="directory-fsync-8.1-to-8.4.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename="directory-fsync-8.1-to-8.4.patch"

diff --git a/src/port/copydir.c b/src/port/copydir.c
index 72fbf36..b057ffa 100644
*** a/src/port/copydir.c
--- b/src/port/copydir.c
*************** copydir(char *fromdir, char *todir, bool
*** 50,55 ****
--- 50,56 ----
  {
  	DIR		   *xldir;
  	struct dirent *xlde;
+ 	int         dirfd;
  	char		fromfile[MAXPGPATH];
  	char		tofile[MAXPGPATH];
  
*************** copydir(char *fromdir, char *todir, bool
*** 91,96 ****
--- 92,116 ----
  	}
  
  	FreeDir(xldir);
+ 
+ 	/*
+ 	 * fsync the directory to make sure data has reached the
+ 	 * disk. While needed by most filesystems, the window got bigger
+ 	 * with newer ones like ext4.
+ 	 */
+ 	dirfd = BasicOpenFile(todir,
+ 	                      O_RDONLY | PG_BINARY,
+ 	                      S_IRUSR | S_IWUSR);
+ 	if(dirfd == -1)
+ 		ereport(ERROR,
+ 		        (errcode_for_file_access(),
+ 		         errmsg("could not open directory for fsync \"%s\": %m", todir)));
+ 
+ 	if(pg_fsync(dirfd) == -1)
+ 		ereport(ERROR,
+ 				(errcode_for_file_access(),
+ 				 errmsg("could not fsync directory \"%s\": %m", todir)));
+ 	close(dirfd);
  }
  
  /*

--Boundary-00=_Ts2cLvEAv/Qj+ul--