Received: from maia.hub.org (unknown [200.46.204.183])
	by mail.postgresql.org (Postfix) with ESMTP id E3AF36325D0;
	Sun,  7 Feb 2010 01:13:27 -0400 (AST)
Received: from mail.postgresql.org ([200.46.204.86])
	by maia.hub.org (mx1.hub.org [200.46.204.183]) (amavisd-maia,
	port 10024)
	with ESMTP id 34199-09; Sun,  7 Feb 2010 05:13:17 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
Received: from mail-fx0-f214.google.com (mail-fx0-f214.google.com
	[209.85.220.214])
	by mail.postgresql.org (Postfix) with ESMTP id EFACE6324D3;
	Sun,  7 Feb 2010 01:13:16 -0400 (AST)
Received: by fxm6 with SMTP id 6so4026685fxm.18
	for <multiple recipients>; Sat, 06 Feb 2010 21:13:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:in-reply-to:references
	:date:message-id:subject:from:to:cc:content-type
	:content-transfer-encoding;
	bh=TEWikZnWHi5NoHWpuw+zLWyeGOLlB5g4Rl+ffGWJIMw=;
	b=iQHQNRwt7mxLgYptuEvh6M6Ri+E8xi6KDJUF/ORZbK7FtbKazqCvJGOvDnpIb0bPFb
	G4gTqD+fLq85GxBXEIY6iHRT4ANXlHBsvylPA5bgQ9cBbGLIRTDfW4bqFcZ+00F7WNkc
	uiUrcpeuie0Jmg7fQQO6pt1dHiZeit5JOO3wE=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=tKQZt4kEdyKlj82hMRAUWQXk6Yq5U1CWhinVg1TJUTmMFGqcjE8Zq8Vo64SezRmxBB
	DxHzuc49ZEt73OmgrhjTtKizVFO7jlzL7DUSyIey7GdSzpOY2963s50WoU8YjlWfjJOA
	IxiIJPJ4mi5vhxE/qWoSEpXr+DzymRdt+0qDA=
MIME-Version: 1.0
Received: by 10.239.187.206 with SMTP id m14mr539503hbh.49.1265519595658;
 Sat,
	06 Feb 2010 21:13:15 -0800 (PST)
In-Reply-To: <201002061304.01682.andres@anarazel.de>
References: <4427a97a0912101235x3ade510blcc00d5ce9883c6b6@mail.gmail.com>
	<4B698605.50506@anarazel.de> <4B6CF822.9010608@2ndquadrant.com>
	<201002061304.01682.andres@anarazel.de>
Date: Sun, 7 Feb 2010 00:13:15 -0500
Message-ID: <603c8f071002062113k262dd8xcd0cdd482d2c6150@mail.gmail.com>
Subject: Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1
	ubuntu karmic slow createdb)
From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Greg Smith <greg@2ndquadrant.com>, Greg Stark <gsstark@mit.edu>,
	Tom Lane <tgl@sss.pgh.pa.us>,
	pgsql-hackers@postgresql.org, pgsql-performance@postgresql.org,
	Michael Clemmons <glassresistor@gmail.com>,
	Hannu Krosing <hannu@2ndquadrant.com>,
	Scott Marlowe <scott.marlowe@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=-2.599 tagged_above=-10 required=5
	tests=BAYES_00=-2.599
X-Spam-Level: 
X-Archive-Number: 201002/108
X-Sequence-Number: 37529

On Sat, Feb 6, 2010 at 7:03 AM, Andres Freund <andres@anarazel.de> wrote:
> On Saturday 06 February 2010 06:03:30 Greg Smith wrote:
>> Andres Freund wrote:
>> > On 02/03/10 14:42, Robert Haas wrote:
>> >> Well, maybe we should start with a discussion of what kernel calls
>> >> you're aware of on different platforms and then we could try to put a=
n
>> >> API around it.
>> >
>> > In linux there is sync_file_range. On newer Posixish systems one can
>> > emulate that with mmap() and msync() (in batches obviously).
>> >
>> > No idea about windows.
>> The effective_io_concurrency feature had proof of concept test programs
>> that worked using AIO, but actually following through on that
>> implementation would require a major restructuring of how the database
>> interacts with the OS in terms of reads and writes of blocks. =A0It look=
s
>> to me like doing something similar to sync_file_range on Windows would
>> be similarly difficult.
> Looking a bit arround it seems one could achieve something approximediate=
ly
> similar to pg_prepare_fsync() by using
> CreateFileMapping && MapViewOfFile && FlushViewOfFile
>
> If I understand it correctly that will flush, but not wait. Unfortunately=
 you
> cant event make it wait, so its not possible to implement sync_file_range=
 or
> similar fully.

Well it seems that what we're trying to implement is more like
it_would_be_nice_if_you_would_start_syncing_this_file_range_but_its_ok_if_y=
ou_dont(),
so maybe that would work.

Anyway, is there something that we can agree on and get committed here
for 9.0, or should we postpone this to 9.1?  It seems simple enough
that we ought to be able to get it done, but we're running out of time
and we don't seem to have a clear vision here yet...

...Robert