From: Abhijit Menon-Sen <ams@2ndQuadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Simon Riggs <simon@2ndQuadrant.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Petr Jelinek <petr@2ndquadrant.com>
Cc: Magnus Hagander <magnus@hagander.net>
Cc: pgsql-hackers@postgresql.org
Subject: Re: 9.6 and fsync=off
Date: Fri, 29 Apr 2016 12:05:14 +0530
Message-ID: <20160429063514.GA630@toroid.org> (raw)
In-Reply-To: <20160428204423.sq2l675d2hamgr6m@alap3.anarazel.de>
References: <CAMsr+YGechDvrFj4xwoBWhJ7rW25Avqo+UqcyqbnS=LTFdGJ4g@mail.gmail.com>
<20160427104307.GA4861@toroid.org>
<CABUevExGL-FWNP0pyCH_9nY==i1ogP-v8XNXpQyDaEp7u4ypCA@mail.gmail.com>
<57209AE7.9020906@2ndquadrant.com>
<24748.1461764666@sss.pgh.pa.us>
<CAMsr+YEy9zj2Nb_n8OFMAf2HpX-RVpb12sWYD2uakPy0Um9kag@mail.gmail.com>
<CANP8+jLAph=Gk1hQy_+o8CvgxuHahZpgh7kqSy2LMq1+OsJS1A@mail.gmail.com>
<20160428204423.sq2l675d2hamgr6m@alap3.anarazel.de>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
At 2016-04-28 13:44:23 -0700, andres@anarazel.de wrote:
>
> Abhijit had a patch implementing automatically running fsync whenever
> reenabled IIRC. Abhijit?
The patch I had written is attached, and it's not quite the same thing.
Here's how I originally described it in response to a question from
Robert:
«In 20150115133245.GG5245@awork2.anarazel.de, Andres explained his
rationale as follows:
«What I am thinking of is that, currently, if you start the
server for initial loading with fsync=off, and then restart it,
you're open to data loss. So when the current config file
setting is changed from off to on, we should fsync the data
directory. Even if there was no crash restart.»
That's what I tried to implement.»
I remember there was some subsequent discussion about it being better to
issue fsync during a checkpoint when we see that its value has changed,
but if I did any work on it (which I have a vague memory of), I can't
find it now. Sorry.
Do you want a patch along those lines now, or is it too late?
-- Abhijit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackersAttachments:0002-Recursively-fsync-PGDATA-on-the-next-restart-after-f.patch (text/x-diff, 3.5K) download
From 1768680b672bcb037446230323cabcf9960f7f9a Mon Sep 17 00:00:00 2001
From: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Fri, 1 May 2015 17:59:51 +0530
Subject: Recursively fsync PGDATA on the next restart after fsync was disabled
Even if we didn't crash, we want to fsync PGDATA on startup if we know
the server ran with fsync=off at some point since the previous restart.
Otherwise, starting the server with fsync=off for initial data loading
and then restarting it opens you to data loss on a power failure after
the restart.
---
src/backend/access/transam/xlog.c | 9 +++++++--
src/bin/pg_controldata/pg_controldata.c | 2 ++
src/include/catalog/pg_control.h | 8 +++++++-
3 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.cindex 084174d..af12992 100644--- a/src/backend/access/transam/xlog.c+++ b/src/backend/access/transam/xlog.c@@ -4823,6 +4823,7 @@ BootStrapXLOG(void)
ControlFile->checkPoint = checkPoint.redo;
ControlFile->checkPointCopy = checkPoint;
ControlFile->unloggedLSN = 1;
+ ControlFile->fsync_disabled = false;
/* Set important parameter values for use when replaying WAL */
ControlFile->MaxConnections = MaxConnections;
@@ -5893,10 +5894,12 @@ StartupXLOG(void)
*/
if (enableFsync &&
- (ControlFile->state != DB_SHUTDOWNED &&- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY))+ (ControlFile->fsync_disabled ||+ (ControlFile->state != DB_SHUTDOWNED &&+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)))
{
fsync_pgdata(data_directory);
+ ControlFile->fsync_disabled = false;
}
if (ControlFile->state == DB_SHUTDOWNED)
@@ -8272,6 +8275,8 @@ CreateCheckPoint(int flags)
/* crash recovery should always recover to the end of WAL */
ControlFile->minRecoveryPoint = InvalidXLogRecPtr;
ControlFile->minRecoveryPointTLI = 0;
+ if (!enableFsync)+ ControlFile->fsync_disabled = true;
/*
* Persist unloggedLSN value. It's reset on crash recovery, so this goes
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.cindex d8cfe5e..e99014f 100644--- a/src/bin/pg_controldata/pg_controldata.c+++ b/src/bin/pg_controldata/pg_controldata.c@@ -290,6 +290,8 @@ main(int argc, char *argv[])
(uint32) ControlFile.backupEndPoint);
printf(_("End-of-backup record required: %s\n"),
ControlFile.backupEndRequired ? _("yes") : _("no"));
+ printf(_("Fsync disabled at runtime: %s\n"),+ ControlFile.fsync_disabled ? _("yes") : _("no"));
printf(_("Current wal_level setting: %s\n"),
wal_level_str(ControlFile.wal_level));
printf(_("Current wal_log_hints setting: %s\n"),
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.hindex 2e4c381..a71d73e 100644--- a/src/include/catalog/pg_control.h+++ b/src/include/catalog/pg_control.h@@ -21,7 +21,7 @@
/* Version identifier for this pg_control format */
-#define PG_CONTROL_VERSION 942+#define PG_CONTROL_VERSION 943
/*
* Body of CheckPoint XLOG records. This is declared here because we keep
@@ -182,6 +182,12 @@ typedef struct ControlFileData
bool track_commit_timestamp;
/*
+ * Indicates whether fsync was ever disabled since the last restart.+ * Tested and set at checkpoints, reset at startup.+ */+ bool fsync_disabled;++ /*
* This data is used to check for hardware-architecture compatibility of
* the database and the backend executable. We need not check endianness
* explicitly, since the pg_control version will surely look wrong to a
--
1.9.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: ams@2ndQuadrant.com, andres@anarazel.de, simon@2ndQuadrant.com, craig@2ndquadrant.com, tgl@sss.pgh.pa.us, petr@2ndquadrant.com, magnus@hagander.net
Subject: Re: 9.6 and fsync=off
In-Reply-To: <20160429063514.GA630@toroid.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox