From: Andres Freund <andres@anarazel.de>
To: Michael Clemmons <glassresistor@gmail.com>
Cc: pgsql-hackers@postgresql.org, Greg Smith <greg@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org, Hannu Krosing <hannu@2ndquadrant.com>
Cc: Scott Marlowe <scott.marlowe@gmail.com>
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Date: Tue, 29 Dec 2009 04:11:14 +0100
Message-ID: <200912290411.15659.andres@anarazel.de> (raw)
In-Reply-To: <4427a97a0912281904qc90817i4edd6852b26dde7f@mail.gmail.com>
References: <4427a97a0912101235x3ade510blcc00d5ce9883c6b6@mail.gmail.com>
<200912290355.38382.andres@anarazel.de>
<4427a97a0912281904qc90817i4edd6852b26dde7f@mail.gmail.com>
On Tuesday 29 December 2009 04:04:06 Michael Clemmons wrote:
> Maybe not crash out but in this situation.
> N=0
> while(N>=0):
> CREATE DATABASE new_db_N;
> Since the fsync is the part which takes the memory and time but is
> happening in the background want the fsyncs pile up in the background
> faster than can be run filling up the memory and stack.
> This is very likely a mistake on my part about how postgres/processes
The difference should not be visible outside the "CREATE DATABASE ..." at all.
Currently the process simplifiedly works like:
------------
for file in source directory:
copy_file(source/file, target/file);
fsync(target/file);
------------
I changed it to:
-------------
for file in source directory:
copy_file(source/file, target/file);
/*please dear kernel, write this out, but dont block*/
posix_fadvise(target/file, FADV_DONTNEED);
for file in source directory:
fsync(target/file);
-------------
If at any point in time there is not enough cache available to cache anything
copy_file() will just have to wait for the kernel to write out the data.
fsync() does not use memory itself.
Andres
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@anarazel.de, glassresistor@gmail.com, greg@2ndquadrant.com, tgl@sss.pgh.pa.us, hannu@2ndquadrant.com, scott.marlowe@gmail.com
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
In-Reply-To: <200912290411.15659.andres@anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox