Received: from maia.hub.org (unknown [200.46.204.183])
	by mail.postgresql.org (Postfix) with ESMTP id 1C0946330C4;
	Mon, 28 Dec 2009 18:55:06 -0400 (AST)
Received: from mail.postgresql.org ([200.46.204.86])
	by maia.hub.org (mx1.hub.org [200.46.204.183]) (amavisd-maia,
	port 10024)
	with ESMTP id 20361-02; Mon, 28 Dec 2009 22:54:55 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from mail.anarazel.de (mail.anarazel.de [217.115.131.40])
	by mail.postgresql.org (Postfix) with ESMTP id 7CFC7632CB2;
	Mon, 28 Dec 2009 18:54:55 -0400 (AST)
Received: from alap2.localnet (unknown [81.163.101.249])
	by mail.anarazel.de (Postfix) with ESMTPSA id 89623448003;
	Mon, 28 Dec 2009 23:54:52 +0100 (CET)
From: Andres Freund <andres@anarazel.de>
To: pgsql-performance@postgresql.org,
 pgsql-hackers@postgresql.org
Subject: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic
	slow createdb)
Date: Mon, 28 Dec 2009 23:54:51 +0100
User-Agent: KMail/1.13.0 (Linux/2.6.33-rc2-andres-00001-g6e0f169; KDE/4.3.85;
	x86_64; ; )
Cc: Michael Clemmons <glassresistor@gmail.com>,
	Hannu Krosing <hannu@2ndquadrant.com>,
	Scott Marlowe <scott.marlowe@gmail.com>
References: <4427a97a0912101235x3ade510blcc00d5ce9883c6b6@mail.gmail.com>
	<4427a97a0912121236g4c06c562u221d638810ed2b6e@mail.gmail.com>
	<200912122138.42404.andres@anarazel.de>
In-Reply-To: <200912122138.42404.andres@anarazel.de>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-15"
Content-Transfer-Encoding: 7bit
Message-Id: <200912282354.51892.andres@anarazel.de>
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=-2.599 tagged_above=-10 required=5
	tests=BAYES_00=-2.599
X-Spam-Level: 
X-Archive-Number: 200912/245
X-Sequence-Number: 36859

On Saturday 12 December 2009 21:38:41 Andres Freund wrote:
> On Saturday 12 December 2009 21:36:27 Michael Clemmons wrote:
> > If ppl think its worth it I'll create a ticket
> Thanks, no need. I will post a patch tomorrow or so.
Well. It was a long day...

Anyway.
In this patch I delay the fsync done in copy_file and simply do a second pass 
over the directory in copy_dir and fsync everything in that pass.
Including the directory - which was not done before and actually might be 
necessary in some cases.
I added a posix_fadvise(..., FADV_DONTNEED) to make it more likely that the 
copied file reaches storage before the fsync. Without the speed benefits were 
quite a bit smaller and essentially random (which seems sensible).

This speeds up CREATE DATABASE from ~9 seconds to something around 0.8s on my 
laptop.  Still slower than with fsync off (~0.25) but quite a worthy 
improvement.

The benefits are obviously bigger if the template database includes anything 
added.


Andres