From: Simon Riggs <simon@2ndQuadrant.com>
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: [PATCHES] Infrastructure changes for recovery (v8)
Date: Wed, 08 Oct 2008 14:34:58 +0100
Message-ID: <1223472898.4747.310.camel@ebony.2ndQuadrant> (raw)
In-Reply-To: <1223468660.4747.294.camel@ebony.2ndQuadrant>
References: <1222815151.4445.1397.camel@ebony.2ndQuadrant>
<48EC9CC5.6060803@enterprisedb.com>
<1223468660.4747.294.camel@ebony.2ndQuadrant>
On Wed, 2008-10-08 at 13:24 +0100, Simon Riggs wrote:
> On Wed, 2008-10-08 at 14:43 +0300, Heikki Linnakangas wrote:
> > Again, perhaps we should do
> > this as a separate patch.
New patch enclosed with other stuff stripped out. 15% lighter...
New thread spawned for bug fix patch. Will resubmit others when we can
be sure they'll not cause patch conflicts.
> OK, I'll strip all of the above out, for separate consideration.
> The form is this
> PM_START startup process performs restartpoints
> transition when database is consistent state
> PM_RECOVERY bgwriter process performs restartpoints
> delicate transition between two states
> PM_RUN bgwriter process performs checkpoints
Above added as comments in patch.
The patch agonises over the two state transitions above. First
transition needs to be exactly correct otherwise we might be using an
inconsistent database during recovery. Second transition is harder
because it isn't just the startup process working alone any more.
The key to understanding it is all in concurrent behaviour. Startup and
bgwriter chat together through bgwriter shared memory and call functions
back and forth between xlog.c and bgwriter.c.
I haven't retested the patch yet, but it passes make check. I'll be
rechecking it later today, starting in about 2 hours time.
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support
Attachments:recovery_infrastruc.v9.patch (text/x-patch, 76.2K) download
Index: src/backend/access/transam/clog.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/clog.c,v
retrieving revision 1.47
diff -c -r1.47 clog.c
*** src/backend/access/transam/clog.c 1 Aug 2008 13:16:08 -0000 1.47
--- src/backend/access/transam/clog.c 8 Oct 2008 12:27:33 -0000
***************
*** 260,265 ****
--- 260,268 ----
/*
* This must be called ONCE during postmaster or standalone-backend startup,
* after StartupXLOG has initialized ShmemVariableCache->nextXid.
+ *+ * We access just a single clog page, so this action is atomic and safe+ * for use if other processes are active during recovery.
*/
void
StartupCLOG(void)
Index: src/backend/access/transam/multixact.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/multixact.c,v
retrieving revision 1.28
diff -c -r1.28 multixact.c
*** src/backend/access/transam/multixact.c 1 Aug 2008 13:16:08 -0000 1.28
--- src/backend/access/transam/multixact.c 8 Oct 2008 12:27:33 -0000
***************
*** 1413,1420 ****
* MultiXactSetNextMXact and/or MultiXactAdvanceNextMXact. Note that we
* may already have replayed WAL data into the SLRU files.
*
! * We don't need any locks here, really; the SLRU locks are taken
! * only because slru.c expects to be called with locks held.
*/
void
StartupMultiXact(void)
--- 1413,1423 ----
* MultiXactSetNextMXact and/or MultiXactAdvanceNextMXact. Note that we
* may already have replayed WAL data into the SLRU files.
*
! * We want this operation to be atomic to ensure that other processes can
! * use MultiXact while we complete recovery. We access one page only from the
! * offset and members buffers, so once locks are acquired they will not be
! * dropped and re-acquired by SLRU code. So we take both locks at start, then
! * hold them all the way to the end.
*/
void
StartupMultiXact(void)
***************
*** 1426,1431 ****
--- 1429,1435 ----
/* Clean up offsets state */
LWLockAcquire(MultiXactOffsetControlLock, LW_EXCLUSIVE);
+ LWLockAcquire(MultiXactMemberControlLock, LW_EXCLUSIVE);
/*
* Initialize our idea of the latest page number.
***************
*** 1452,1461 ****
MultiXactOffsetCtl->shared->page_dirty[slotno] = true;
}
- LWLockRelease(MultiXactOffsetControlLock);-
/* And the same for members */
- LWLockAcquire(MultiXactMemberControlLock, LW_EXCLUSIVE);
/*
* Initialize our idea of the latest page number.
--- 1456,1462 ----
***************
*** 1483,1488 ****
--- 1484,1490 ----
}
LWLockRelease(MultiXactMemberControlLock);
+ LWLockRelease(MultiXactOffsetControlLock);
/*
* Initialize lastTruncationPoint to invalid, ensuring that the first
***************
*** 1543,1549 ****
* SimpleLruTruncate would get confused. It seems best not to risk
* removing any data during recovery anyway, so don't truncate.
*/
! if (!InRecovery)
TruncateMultiXact();
TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_DONE(true);
--- 1545,1551 ----
* SimpleLruTruncate would get confused. It seems best not to risk
* removing any data during recovery anyway, so don't truncate.
*/
! if (!IsRecoveryProcessingMode())
TruncateMultiXact();
TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_DONE(true);
Index: src/backend/access/transam/subtrans.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/subtrans.c,v
retrieving revision 1.23
diff -c -r1.23 subtrans.c
*** src/backend/access/transam/subtrans.c 1 Aug 2008 13:16:08 -0000 1.23
--- src/backend/access/transam/subtrans.c 8 Oct 2008 12:27:33 -0000
***************
*** 226,231 ****
--- 226,234 ----
*
* oldestActiveXID is the oldest XID of any prepared transaction, or nextXid
* if there are none.
+ *+ * Note that this is not atomic and is not yet safe to perform while other+ * processes might access subtrans.
*/
void
StartupSUBTRANS(TransactionId oldestActiveXID)
Index: src/backend/access/transam/xact.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/xact.c,v
retrieving revision 1.265
diff -c -r1.265 xact.c
*** src/backend/access/transam/xact.c 11 Aug 2008 11:05:10 -0000 1.265
--- src/backend/access/transam/xact.c 8 Oct 2008 12:27:33 -0000
***************
*** 393,398 ****
--- 393,401 ----
bool isSubXact = (s->parent != NULL);
ResourceOwner currentOwner;
+ if (IsRecoveryProcessingMode())+ elog(FATAL, "cannot assign TransactionIds during recovery");+
/* Assert that caller didn't screw up */
Assert(!TransactionIdIsValid(s->transactionId));
Assert(s->state == TRANS_INPROGRESS);
Index: src/backend/access/transam/xlog.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.319
diff -c -r1.319 xlog.c
*** src/backend/access/transam/xlog.c 23 Sep 2008 09:20:35 -0000 1.319
--- src/backend/access/transam/xlog.c 8 Oct 2008 13:23:16 -0000
***************
*** 113,119 ****
/*
* ThisTimeLineID will be same in all backends --- it identifies current
! * WAL timeline for the database system.
*/
TimeLineID ThisTimeLineID = 0;
--- 113,120 ----
/*
* ThisTimeLineID will be same in all backends --- it identifies current
! * WAL timeline for the database system. Zero is always a bug, so we
! * start with that to allow us to spot any errors.
*/
TimeLineID ThisTimeLineID = 0;
***************
*** 123,128 ****
--- 124,133 ----
/* Are we recovering using offline XLOG archives? */
static bool InArchiveRecovery = false;
+ /* Local copy of shared RecoveryProcessingMode state */+ static bool LocalRecoveryProcessingMode = true;+ static bool knownProcessingMode = false;+
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
***************
*** 141,146 ****
--- 146,154 ----
static TimestampTz recoveryStopTime;
static bool recoveryStopAfter;
+ /* is the database proven consistent yet? */+ bool reachedSafeStartPoint = false;+
/*
* During normal operation, the only timeline we care about is ThisTimeLineID.
* During recovery, however, things are more complicated. To simplify life
***************
*** 240,249 ****
* ControlFileLock: must be held to read/update control file or create
* new log file.
*
! * CheckpointLock: must be held to do a checkpoint (ensures only one
! * checkpointer at a time; currently, with all checkpoints done by the
! * bgwriter, this is just pro forma).
! *
*----------
*/
--- 248,277 ----
* ControlFileLock: must be held to read/update control file or create
* new log file.
*
! * CheckpointLock: must be held to do a checkpoint or restartpoint, ensuring
! * we get just one of those at any time. In 8.4+ recovery, both startup and
! * bgwriter processes may take restartpoints, so this locking must be strict
! * to ensure there are no mistakes.
! *
! * In 8.4 we progress through a number of states at startup. Initially, the
! * postmaster is in PM_STARTUP state and spawns the Startup process. We then
! * progress until the database is in a consistent state, then if we are in
! * InArchiveRecovery we go into PM_RECOVERY state. The bgwriter then starts
! * up and takes over responsibility for performing restartpoints. We then
! * progress until the end of recovery when we enter PM_RUN state upon
! * termination of the Startup process. In summary:
! *
! * PM_STARTUP state: Startup process performs restartpoints
! * PM_RECOVERY state: bgwriter process performs restartpoints
! * PM_RUN state: bgwriter process performs checkpoints
! *
! * These transitions are fairly delicate, with many things that need to
! * happen at the same time in order to change state successfully throughout
! * the system. Changing PM_STARTUP to PM_RECOVERY only occurs when we can
! * prove the databases are in a consistent state. Changing from PM_RECOVERY
! * to PM_RUN happens whenever recovery ends, which could be forced upon us
! * externally or it can occur becasue of damage or termination of the WAL
! * sequence.
*----------
*/
***************
*** 285,295 ****
--- 313,330 ----
/*
* Total shared-memory state for XLOG.
+ *+ * This small structure is accessed by many backends, so we take care to+ * pad out the parts of the structure so they can be accessed by separate+ * CPUs without causing false sharing cache flushes. Padding is generous+ * to allow for a wide variety of CPU architectures.
*/
+ #define XLOGCTL_BUFFER_SPACING 128
typedef struct XLogCtlData
{
/* Protected by WALInsertLock: */
XLogCtlInsert Insert;
+ char InsertPadding[XLOGCTL_BUFFER_SPACING - sizeof(XLogCtlInsert)];
/* Protected by info_lck: */
XLogwrtRqst LogwrtRqst;
***************
*** 297,305 ****
--- 332,347 ----
uint32 ckptXidEpoch; /* nextXID & epoch of latest checkpoint */
TransactionId ckptXid;
XLogRecPtr asyncCommitLSN; /* LSN of newest async commit */
+ /* add data structure padding for above info_lck declarations */+ char InfoPadding[XLOGCTL_BUFFER_SPACING - sizeof(XLogwrtRqst) + - sizeof(XLogwrtResult)+ - sizeof(uint32)+ - sizeof(TransactionId)+ - sizeof(XLogRecPtr)];
/* Protected by WALWriteLock: */
XLogCtlWrite Write;
+ char WritePadding[XLOGCTL_BUFFER_SPACING - sizeof(XLogCtlWrite)];
/*
* These values do not change after startup, although the pointed-to pages
***************
*** 311,316 ****
--- 353,376 ----
int XLogCacheBlck; /* highest allocated xlog buffer index */
TimeLineID ThisTimeLineID;
+ /*+ * IsRecoveryProcessingMode shows whether the postmaster is in a+ * postmaster state earlier than PM_RUN, or not. This is a globally+ * accessible state to allow EXEC_BACKEND case.+ *+ * We also retain a local state variable InRecovery. InRecovery=true+ * means the code is being executed by Startup process and therefore+ * always during Recovery Processing Mode. This allows us to identify+ * code executed *during* Recovery Processing Mode but not necessarily+ * by Startup process itself.+ *+ * Protected by mode_lck+ */+ bool SharedRecoveryProcessingMode;+ slock_t mode_lck;+ + char InfoLockPadding[XLOGCTL_BUFFER_SPACING];+
slock_t info_lck; /* locks shared variables shown above */
} XLogCtlData;
***************
*** 397,404 ****
--- 457,466 ----
static void readRecoveryCommandFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
+ static void exitRecovery(void);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
+ static XLogRecPtr GetRedoLocationForCheckpoint(void);
static bool XLogCheckBuffer(XLogRecData *rdata, bool doPageWrites,
XLogRecPtr *lsn, BkpBlock *bkpb);
***************
*** 480,485 ****
--- 542,552 ----
bool updrqst;
bool doPageWrites;
bool isLogSwitch = (rmid == RM_XLOG_ID && info == XLOG_SWITCH);
+ bool isRecoveryEnd = (rmid == RM_XLOG_ID && info == XLOG_RECOVERY_END);+ + /* cross-check on whether we should be here or not */+ if (IsRecoveryProcessingMode() && !isRecoveryEnd)+ elog(FATAL, "cannot make new WAL entries during recovery");
/* info's high bits are reserved for use by me */
if (info & XLR_INFO_MASK)
***************
*** 1720,1727 ****
XLogRecPtr WriteRqstPtr;
XLogwrtRqst WriteRqst;
! /* Disabled during REDO */
! if (InRedo)
return;
/* Quick exit if already known flushed */
--- 1787,1793 ----
XLogRecPtr WriteRqstPtr;
XLogwrtRqst WriteRqst;
! if (IsRecoveryProcessingMode())
return;
/* Quick exit if already known flushed */
***************
*** 1809,1817 ****
* the bad page is encountered again during recovery then we would be
* unable to restart the database at all! (This scenario has actually
* happened in the field several times with 7.1 releases. Note that we
! * cannot get here while InRedo is true, but if the bad page is brought in
! * and marked dirty during recovery then CreateCheckPoint will try to
! * flush it at the end of recovery.)
*
* The current approach is to ERROR under normal conditions, but only
* WARNING during recovery, so that the system can be brought up even if
--- 1875,1883 ----
* the bad page is encountered again during recovery then we would be
* unable to restart the database at all! (This scenario has actually
* happened in the field several times with 7.1 releases. Note that we
! * cannot get here while IsRecoveryProcessingMode(), but if the bad page is
! * brought in and marked dirty during recovery then if a checkpoint were
! * performed at the end of recovery it will try to flush it.
*
* The current approach is to ERROR under normal conditions, but only
* WARNING during recovery, so that the system can be brought up even if
***************
*** 1821,1827 ****
* and so we will not force a restart for a bad LSN on a data page.
*/
if (XLByteLT(LogwrtResult.Flush, record))
! elog(InRecovery ? WARNING : ERROR,
"xlog flush request %X/%X is not satisfied --- flushed only to %X/%X",
record.xlogid, record.xrecoff,
LogwrtResult.Flush.xlogid, LogwrtResult.Flush.xrecoff);
--- 1887,1893 ----
* and so we will not force a restart for a bad LSN on a data page.
*/
if (XLByteLT(LogwrtResult.Flush, record))
! elog(ERROR,
"xlog flush request %X/%X is not satisfied --- flushed only to %X/%X",
record.xlogid, record.xrecoff,
LogwrtResult.Flush.xlogid, LogwrtResult.Flush.xrecoff);
***************
*** 2094,2100 ****
unlink(tmppath);
}
! elog(DEBUG2, "done creating and filling new WAL file");
/* Set flag to tell caller there was no existent file */
*use_existent = false;
--- 2160,2167 ----
unlink(tmppath);
}
! XLogFileName(tmppath, ThisTimeLineID, log, seg);
! elog(DEBUG2, "done creating and filling new WAL file %s", tmppath);
/* Set flag to tell caller there was no existent file */
*use_existent = false;
***************
*** 2400,2405 ****
--- 2467,2494 ----
xlogfname);
set_ps_display(activitymsg, false);
+ /* + * Calculate and write out a new safeStartPoint. This defines+ * the latest LSN that might appear on-disk while we apply+ * the WAL records in this file. If we crash during recovery+ * we must reach this point again before we can prove+ * database consistency. Not a restartpoint! Restart points+ * define where we should start recovery from, if we crash.+ */+ if (InArchiveRecovery)+ {+ uint32 nextLog = log;+ uint32 nextSeg = seg;+ + NextLogSeg(nextLog, nextSeg);+ + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);+ ControlFile->minSafeStartPoint.xlogid = nextLog;+ ControlFile->minSafeStartPoint.xrecoff = nextSeg * XLogSegSize;+ UpdateControlFile();+ LWLockRelease(ControlFileLock);+ }+
return fd;
}
if (errno != ENOENT) /* unexpected failure? */
***************
*** 4228,4233 ****
--- 4317,4323 ----
XLogCtl->XLogCacheBlck = XLOGbuffers - 1;
XLogCtl->Insert.currpage = (XLogPageHeader) (XLogCtl->pages);
SpinLockInit(&XLogCtl->info_lck);
+ SpinLockInit(&XLogCtl->mode_lck);
/*
* If we are not in bootstrap mode, pg_control should already exist. Read
***************
*** 4538,4549 ****
* does nothing if a recovery_target is not also set
*/
if (!parse_bool(tok2, &recoveryLogRestartpoints))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"log_restartpoints\" requires a Boolean value")));
ereport(LOG,
! (errmsg("log_restartpoints = %s", tok2)));
! }
else
ereport(FATAL,
(errmsg("unrecognized recovery parameter \"%s\"",
--- 4628,4639 ----
* does nothing if a recovery_target is not also set
*/
if (!parse_bool(tok2, &recoveryLogRestartpoints))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"log_restartpoints\" requires a Boolean value")));
ereport(LOG,
! (errmsg("log_restartpoints = %s", tok2)));
! }
else
ereport(FATAL,
(errmsg("unrecognized recovery parameter \"%s\"",
***************
*** 4678,4692 ****
unlink(recoveryPath); /* ignore any error */
/*
! * Rename the config file out of the way, so that we don't accidentally
! * re-enter archive recovery mode in a subsequent crash.
*/
- unlink(RECOVERY_COMMAND_DONE);- if (rename(RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE) != 0)- ereport(FATAL,- (errcode_for_file_access(),- errmsg("could not rename file \"%s\" to \"%s\": %m",- RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
--- 4768,4780 ----
unlink(recoveryPath); /* ignore any error */
/*
! * As of 8.4 we no longer rename the recovery.conf file out of the
! * way until after we have performed a full checkpoint. This ensures
! * that any crash between now and the end of the checkpoint does not
! * attempt to restart from a WAL file that is no longer available to us.
! * As soon as we remove recovery.conf we lose our recovery_command and
! * cannot reaccess WAL files from the archive.
*/
ereport(LOG,
(errmsg("archive recovery complete")));
***************
*** 4813,4818 ****
--- 4901,4907 ----
CheckPoint checkPoint;
bool wasShutdown;
bool reachedStopPoint = false;
+ bool performedRecovery = false;
bool haveBackupLabel = false;
XLogRecPtr RecPtr,
LastRec,
***************
*** 4825,4830 ****
--- 4914,4921 ----
uint32 freespace;
TransactionId oldestActiveXID;
+ XLogCtl->SharedRecoveryProcessingMode = true;+
/*
* Read control file and check XLOG status looks valid.
*
***************
*** 5038,5046 ****
--- 5129,5143 ----
if (minRecoveryLoc.xlogid != 0 || minRecoveryLoc.xrecoff != 0)
ControlFile->minRecoveryPoint = minRecoveryLoc;
ControlFile->time = (pg_time_t) time(NULL);
+ /* No need to hold ControlFileLock yet, we aren't up far enough */
UpdateControlFile();
/*
+ * Reset pgstat data, because it may be invalid after recovery.+ */+ pgstat_reset_all();+ + /*
* If there was a backup label file, it's done its job and the info
* has now been propagated into pg_control. We must get rid of the
* label file so that if we crash during recovery, we'll pick up at
***************
*** 5150,5155 ****
--- 5247,5278 ----
LastRec = ReadRecPtr;
+ /*+ * Have we reached our safe starting point? If so, we can+ * signal Postmaster to enter consistent recovery mode.+ *+ * There are two point in the log we must pass. The first is+ * the minRecoveryPoint, which is the LSN at the time the+ * base backup was taken that we are about to rollfoward from.+ * If recovery has ever crashed or was stopped there is + * another point also: minSafeStartPoint, which we know the+ * latest LSN that recovery could have reached prior to crash.+ */+ if (!reachedSafeStartPoint && + XLByteLE(ControlFile->minSafeStartPoint, EndRecPtr) && + XLByteLE(ControlFile->minRecoveryPoint, EndRecPtr))+ {+ reachedSafeStartPoint = true;+ if (InArchiveRecovery)+ {+ ereport(LOG,+ (errmsg("consistent recovery state reached at %X/%X",+ EndRecPtr.xlogid, EndRecPtr.xrecoff)));+ if (IsUnderPostmaster)+ SendPostmasterSignal(PMSIGNAL_RECOVERY_START);+ }+ }+
record = ReadRecord(NULL, LOG);
} while (record != NULL && recoveryContinue);
***************
*** 5171,5176 ****
--- 5294,5300 ----
/* there are no WAL records following the checkpoint */
ereport(LOG,
(errmsg("redo is not required")));
+ reachedSafeStartPoint = true;
}
}
***************
*** 5184,5192 ****
/*
* Complain if we did not roll forward far enough to render the backup
! * dump consistent.
*/
! if (XLByteLT(EndOfLog, ControlFile->minRecoveryPoint))
{
if (reachedStopPoint) /* stopped because of stop request */
ereport(FATAL,
--- 5308,5316 ----
/*
* Complain if we did not roll forward far enough to render the backup
! * dump consistent and start safely.
*/
! if (InRecovery && !reachedSafeStartPoint)
{
if (reachedStopPoint) /* stopped because of stop request */
ereport(FATAL,
***************
*** 5308,5346 ****
XLogCheckInvalidPages();
/*
! * Reset pgstat data, because it may be invalid after recovery.
*/
! pgstat_reset_all();
! /*
! * Perform a checkpoint to update all our recovery activity to disk.
! *
! * Note that we write a shutdown checkpoint rather than an on-line
! * one. This is not particularly critical, but since we may be
! * assigning a new TLI, using a shutdown checkpoint allows us to have
! * the rule that TLI only changes in shutdown checkpoints, which
! * allows some extra error checking in xlog_redo.
! */
! CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
}
- /*- * Preallocate additional log files, if wanted.- */- PreallocXlogFiles(EndOfLog);- - /*- * Okay, we're officially UP.- */- InRecovery = false;- - ControlFile->state = DB_IN_PRODUCTION;- ControlFile->time = (pg_time_t) time(NULL);- UpdateControlFile();- - /* start the archive_timeout timer running */- XLogCtl->Write.lastSegSwitchTime = ControlFile->time;-
/* initialize shared-memory copy of latest checkpoint XID/epoch */
XLogCtl->ckptXidEpoch = ControlFile->checkPointCopy.nextXidEpoch;
XLogCtl->ckptXid = ControlFile->checkPointCopy.nextXid;
--- 5432,5445 ----
XLogCheckInvalidPages();
/*
! * Finally exit recovery and mark that in WAL. Pre-8.4 we wrote
! * a shutdown checkpoint here, but we ask bgwriter to do that now.
*/
! exitRecovery();
! performedRecovery = true;
}
/* initialize shared-memory copy of latest checkpoint XID/epoch */
XLogCtl->ckptXidEpoch = ControlFile->checkPointCopy.nextXidEpoch;
XLogCtl->ckptXid = ControlFile->checkPointCopy.nextXid;
***************
*** 5374,5379 ****
--- 5473,5561 ----
readRecordBuf = NULL;
readRecordBufSize = 0;
}
+ + /*+ * Prior to 8.4 we wrote a Shutdown Checkpoint at the end of recovery.+ * This could add minutes to the startup time, so we want bgwriter+ * to perform it. This then frees the Startup process to complete so we can+ * allow transactions and WAL inserts. We still write a checkpoint, but+ * it will be an online checkpoint. Online checkpoints have a redo+ * location that can be prior to the actual checkpoint record. So we want+ * to derive that redo location *before* we let anybody else write WAL,+ * otherwise we might miss some WAL records if we crash.+ */+ if (performedRecovery)+ {+ XLogRecPtr redo;+ + /* + * We must grab the pointer before anybody writes WAL + */+ redo = GetRedoLocationForCheckpoint();+ + /* + * Tell the bgwriter+ */+ SetRedoLocationForArchiveCheckpoint(redo);+ + /*+ * Okay, we can come up now. Allow others to write WAL.+ */+ XLogCtl->SharedRecoveryProcessingMode = false;+ + /*+ * Now request checkpoint+ */+ RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_IMMEDIATE);+ }+ else+ {+ /*+ * No recovery, so lets just get on with it. + */+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);+ ControlFile->state = DB_IN_PRODUCTION;+ ControlFile->time = (pg_time_t) time(NULL);+ UpdateControlFile();+ LWLockRelease(ControlFileLock);+ + /*+ * Okay, we're officially UP.+ */+ XLogCtl->SharedRecoveryProcessingMode = false;+ }+ + /* start the archive_timeout timer running */+ XLogCtl->Write.lastSegSwitchTime = (pg_time_t) time(NULL);+ + }+ + /*+ * IsRecoveryProcessingMode()+ *+ * Fast test for whether we're still in recovery or not. We test the shared+ * state each time only until we leave recovery mode. After that we never+ * look again, relying upon the settings of our local state variables. This+ * is designed to avoid the need for a separate initialisation step.+ */+ bool+ IsRecoveryProcessingMode(void)+ {+ if (knownProcessingMode && !LocalRecoveryProcessingMode)+ return false;+ + {+ /* use volatile pointer to prevent code rearrangement */+ volatile XLogCtlData *xlogctl = XLogCtl;+ + SpinLockAcquire(&xlogctl->mode_lck);+ LocalRecoveryProcessingMode = XLogCtl->SharedRecoveryProcessingMode;+ SpinLockRelease(&xlogctl->mode_lck);+ }+ + knownProcessingMode = true;+ + return LocalRecoveryProcessingMode;
}
/*
***************
*** 5631,5650 ****
static void
LogCheckpointStart(int flags)
{
! elog(LOG, "checkpoint starting:%s%s%s%s%s%s",
! (flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
! (flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
! (flags & CHECKPOINT_FORCE) ? " force" : "",
! (flags & CHECKPOINT_WAIT) ? " wait" : "",
! (flags & CHECKPOINT_CAUSE_XLOG) ? " xlog" : "",
! (flags & CHECKPOINT_CAUSE_TIME) ? " time" : "");
}
/*
* Log end of a checkpoint.
*/
static void
! LogCheckpointEnd(void)
{
long write_secs,
sync_secs,
--- 5813,5836 ----
static void
LogCheckpointStart(int flags)
{
! if (flags & CHECKPOINT_RESTARTPOINT)
! elog(LOG, "restartpoint starting:%s",
! (flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "");
! else
! elog(LOG, "checkpoint starting:%s%s%s%s%s%s",
! (flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
! (flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
! (flags & CHECKPOINT_FORCE) ? " force" : "",
! (flags & CHECKPOINT_WAIT) ? " wait" : "",
! (flags & CHECKPOINT_CAUSE_XLOG) ? " xlog" : "",
! (flags & CHECKPOINT_CAUSE_TIME) ? " time" : "");
}
/*
* Log end of a checkpoint.
*/
static void
! LogCheckpointEnd(int flags)
{
long write_secs,
sync_secs,
***************
*** 5667,5683 ****
CheckpointStats.ckpt_sync_end_t,
&sync_secs, &sync_usecs);
! elog(LOG, "checkpoint complete: wrote %d buffers (%.1f%%); "
! "%d transaction log file(s) added, %d removed, %d recycled; "
! "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s",
! CheckpointStats.ckpt_bufs_written,
! (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers,
! CheckpointStats.ckpt_segs_added,
! CheckpointStats.ckpt_segs_removed,
! CheckpointStats.ckpt_segs_recycled,
! write_secs, write_usecs / 1000,
! sync_secs, sync_usecs / 1000,
! total_secs, total_usecs / 1000);
}
/*
--- 5853,5878 ----
CheckpointStats.ckpt_sync_end_t,
&sync_secs, &sync_usecs);
! if (flags & CHECKPOINT_RESTARTPOINT)
! elog(LOG, "restartpoint complete: wrote %d buffers (%.1f%%); "
! "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s",
! CheckpointStats.ckpt_bufs_written,
! (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers,
! write_secs, write_usecs / 1000,
! sync_secs, sync_usecs / 1000,
! total_secs, total_usecs / 1000);
! else
! elog(LOG, "checkpoint complete: wrote %d buffers (%.1f%%); "
! "%d transaction log file(s) added, %d removed, %d recycled; "
! "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s",
! CheckpointStats.ckpt_bufs_written,
! (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers,
! CheckpointStats.ckpt_segs_added,
! CheckpointStats.ckpt_segs_removed,
! CheckpointStats.ckpt_segs_recycled,
! write_secs, write_usecs / 1000,
! sync_secs, sync_usecs / 1000,
! total_secs, total_usecs / 1000);
}
/*
***************
*** 5702,5718 ****
XLogRecPtr recptr;
XLogCtlInsert *Insert = &XLogCtl->Insert;
XLogRecData rdata;
- uint32 freespace;
uint32 _logId;
uint32 _logSeg;
TransactionId *inCommitXids;
int nInCommit;
/*
* Acquire CheckpointLock to ensure only one checkpoint happens at a time.
! * (This is just pro forma, since in the present system structure there is
! * only one process that is allowed to issue checkpoints at any given
! * time.)
*/
LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
--- 5897,5912 ----
XLogRecPtr recptr;
XLogCtlInsert *Insert = &XLogCtl->Insert;
XLogRecData rdata;
uint32 _logId;
uint32 _logSeg;
TransactionId *inCommitXids;
int nInCommit;
+ bool leavingArchiveRecovery = false;
/*
* Acquire CheckpointLock to ensure only one checkpoint happens at a time.
! * That shouldn't be happening, but checkpoints are an important aspect
! * of our resilience, so we take no chances.
*/
LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
***************
*** 5727,5741 ****
--- 5921,5944 ----
CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
/*
+ * Find out if this is the first checkpoint after archive recovery.+ */+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);+ leavingArchiveRecovery = (ControlFile->state == DB_IN_ARCHIVE_RECOVERY);+ LWLockRelease(ControlFileLock);+ + /*
* Use a critical section to force system panic if we have trouble.
*/
START_CRIT_SECTION();
if (shutdown)
{
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
ControlFile->state = DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
+ LWLockRelease(ControlFileLock);
}
/*
***************
*** 5750,5840 ****
checkPoint.ThisTimeLineID = ThisTimeLineID;
checkPoint.time = (pg_time_t) time(NULL);
! /*
! * We must hold WALInsertLock while examining insert state to determine
! * the checkpoint REDO pointer.
! */
! LWLockAcquire(WALInsertLock, LW_EXCLUSIVE);
!
! /*
! * If this isn't a shutdown or forced checkpoint, and we have not inserted
! * any XLOG records since the start of the last checkpoint, skip the
! * checkpoint. The idea here is to avoid inserting duplicate checkpoints
! * when the system is idle. That wastes log space, and more importantly it
! * exposes us to possible loss of both current and previous checkpoint
! * records if the machine crashes just as we're writing the update.
! * (Perhaps it'd make even more sense to checkpoint only when the previous
! * checkpoint record is in a different xlog page?)
! *
! * We have to make two tests to determine that nothing has happened since
! * the start of the last checkpoint: current insertion point must match
! * the end of the last checkpoint record, and its redo pointer must point
! * to itself.
! */
! if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_FORCE)) == 0)
{
! XLogRecPtr curInsert;
! INSERT_RECPTR(curInsert, Insert, Insert->curridx);
! if (curInsert.xlogid == ControlFile->checkPoint.xlogid &&
! curInsert.xrecoff == ControlFile->checkPoint.xrecoff +
! MAXALIGN(SizeOfXLogRecord + sizeof(CheckPoint)) &&
! ControlFile->checkPoint.xlogid ==
! ControlFile->checkPointCopy.redo.xlogid &&
! ControlFile->checkPoint.xrecoff ==
! ControlFile->checkPointCopy.redo.xrecoff)
{
! LWLockRelease(WALInsertLock);
! LWLockRelease(CheckpointLock);
! END_CRIT_SECTION();
! return;
! }
! }
! /*
! * Compute new REDO record ptr = location of next XLOG record.
! *
! * NB: this is NOT necessarily where the checkpoint record itself will be,
! * since other backends may insert more XLOG records while we're off doing
! * the buffer flush work. Those XLOG records are logically after the
! * checkpoint, even though physically before it. Got that?
! */
! freespace = INSERT_FREESPACE(Insert);
! if (freespace < SizeOfXLogRecord)
! {
! (void) AdvanceXLInsertBuffer(false);
! /* OK to ignore update return flag, since we will do flush anyway */
! freespace = INSERT_FREESPACE(Insert);
! }
! INSERT_RECPTR(checkPoint.redo, Insert, Insert->curridx);
! /*
! * Here we update the shared RedoRecPtr for future XLogInsert calls; this
! * must be done while holding the insert lock AND the info_lck.
! *
! * Note: if we fail to complete the checkpoint, RedoRecPtr will be left
! * pointing past where it really needs to point. This is okay; the only
! * consequence is that XLogInsert might back up whole buffers that it
! * didn't really need to. We can't postpone advancing RedoRecPtr because
! * XLogInserts that happen while we are dumping buffers must assume that
! * their buffer changes are not included in the checkpoint.
! */
! {
! /* use volatile pointer to prevent code rearrangement */
! volatile XLogCtlData *xlogctl = XLogCtl;
! SpinLockAcquire(&xlogctl->info_lck);
! RedoRecPtr = xlogctl->Insert.RedoRecPtr = checkPoint.redo;
! SpinLockRelease(&xlogctl->info_lck);
}
/*
- * Now we can release WAL insert lock, allowing other xacts to proceed- * while we are flushing disk buffers.- */- LWLockRelease(WALInsertLock);- - /*
* If enabled, log checkpoint start. We postpone this until now so as not
* to log anything if we decided to skip the checkpoint.
*/
--- 5953,6021 ----
checkPoint.ThisTimeLineID = ThisTimeLineID;
checkPoint.time = (pg_time_t) time(NULL);
! if (leavingArchiveRecovery)
! checkPoint.redo = GetRedoLocationForArchiveCheckpoint();
! else
{
! /*
! * We must hold WALInsertLock while examining insert state to determine
! * the checkpoint REDO pointer.
! */
! LWLockAcquire(WALInsertLock, LW_EXCLUSIVE);
! /*
! * If this isn't a shutdown or forced checkpoint, and we have not inserted
! * any XLOG records since the start of the last checkpoint, skip the
! * checkpoint. The idea here is to avoid inserting duplicate checkpoints
! * when the system is idle. That wastes log space, and more importantly it
! * exposes us to possible loss of both current and previous checkpoint
! * records if the machine crashes just as we're writing the update.
! * (Perhaps it'd make even more sense to checkpoint only when the previous
! * checkpoint record is in a different xlog page?)
! *
! * We have to make two tests to determine that nothing has happened since
! * the start of the last checkpoint: current insertion point must match
! * the end of the last checkpoint record, and its redo pointer must point
! * to itself.
! */
! if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_FORCE)) == 0)
{
! XLogRecPtr curInsert;
! INSERT_RECPTR(curInsert, Insert, Insert->curridx);
! if (curInsert.xlogid == ControlFile->checkPoint.xlogid &&
! curInsert.xrecoff == ControlFile->checkPoint.xrecoff +
! MAXALIGN(SizeOfXLogRecord + sizeof(CheckPoint)) &&
! ControlFile->checkPoint.xlogid ==
! ControlFile->checkPointCopy.redo.xlogid &&
! ControlFile->checkPoint.xrecoff ==
! ControlFile->checkPointCopy.redo.xrecoff)
! {
! LWLockRelease(WALInsertLock);
! LWLockRelease(CheckpointLock);
! END_CRIT_SECTION();
! return;
! }
! }
! /*
! * Compute new REDO record ptr = location of next XLOG record.
! *
! * NB: this is NOT necessarily where the checkpoint record itself will be,
! * since other backends may insert more XLOG records while we're off doing
! * the buffer flush work. Those XLOG records are logically after the
! * checkpoint, even though physically before it. Got that?
! */
! checkPoint.redo = GetRedoLocationForCheckpoint();
! /*
! * Now we can release WAL insert lock, allowing other xacts to proceed
! * while we are flushing disk buffers.
! */
! LWLockRelease(WALInsertLock);
}
/*
* If enabled, log checkpoint start. We postpone this until now so as not
* to log anything if we decided to skip the checkpoint.
*/
***************
*** 5941,5958 ****
XLByteToSeg(ControlFile->checkPointCopy.redo, _logId, _logSeg);
/*
! * Update the control file.
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
ControlFile->state = DB_SHUTDOWNED;
ControlFile->prevCheckPoint = ControlFile->checkPoint;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
/* Update shared-memory copy of checkpoint XID/epoch */
{
/* use volatile pointer to prevent code rearrangement */
--- 6122,6164 ----
XLByteToSeg(ControlFile->checkPointCopy.redo, _logId, _logSeg);
/*
! * Update the control file. In 8.4, this routine becomes the primary
! * point for recording changes of state in the control file at the
! * end of recovery. Postmaster state already shows us being in
! * normal running mode, but it is only after this point that we
! * are completely free of reperforming a recovery if we crash. Note
! * that this is executed by bgwriter after the death of Startup process.
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+
if (shutdown)
ControlFile->state = DB_SHUTDOWNED;
+ else+ ControlFile->state = DB_IN_PRODUCTION;+
ControlFile->prevCheckPoint = ControlFile->checkPoint;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
+
LWLockRelease(ControlFileLock);
+ if (leavingArchiveRecovery)+ {+ /*+ * Rename the config file out of the way, so that we don't accidentally+ * re-enter archive recovery mode in a subsequent crash. Prior to+ * 8.4 this step was performed at end of exitArchiveRecovery().+ */+ unlink(RECOVERY_COMMAND_DONE);+ if (rename(RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE) != 0)+ ereport(ERROR,+ (errcode_for_file_access(),+ errmsg("could not rename file \"%s\" to \"%s\": %m",+ RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE)));+ }+
/* Update shared-memory copy of checkpoint XID/epoch */
{
/* use volatile pointer to prevent code rearrangement */
***************
*** 5999,6014 ****
* in subtrans.c). During recovery, though, we mustn't do this because
* StartupSUBTRANS hasn't been called yet.
*/
! if (!InRecovery)
! TruncateSUBTRANS(GetOldestXmin(true, false));
/* All real work is done, but log before releasing lock. */
if (log_checkpoints)
! LogCheckpointEnd();
LWLockRelease(CheckpointLock);
}
/*
* Flush all data in shared memory to disk, and fsync
*
--- 6205,6264 ----
* in subtrans.c). During recovery, though, we mustn't do this because
* StartupSUBTRANS hasn't been called yet.
*/
! TruncateSUBTRANS(GetOldestXmin(true, false));
/* All real work is done, but log before releasing lock. */
if (log_checkpoints)
! LogCheckpointEnd(flags);
LWLockRelease(CheckpointLock);
}
+ /* + * GetRedoLocationForCheckpoint()+ *+ * When !IsRecoveryProcessingMode() this must be called while holding + * WALInsertLock().+ */+ static XLogRecPtr+ GetRedoLocationForCheckpoint()+ {+ XLogCtlInsert *Insert = &XLogCtl->Insert;+ uint32 freespace;+ XLogRecPtr redo;+ + freespace = INSERT_FREESPACE(Insert);+ if (freespace < SizeOfXLogRecord)+ {+ (void) AdvanceXLInsertBuffer(false);+ /* OK to ignore update return flag, since we will do flush anyway */+ freespace = INSERT_FREESPACE(Insert);+ }+ INSERT_RECPTR(redo, Insert, Insert->curridx);+ + /*+ * Here we update the shared RedoRecPtr for future XLogInsert calls; this+ * must be done while holding the insert lock AND the info_lck.+ *+ * Note: if we fail to complete the checkpoint, RedoRecPtr will be left+ * pointing past where it really needs to point. This is okay; the only+ * consequence is that XLogInsert might back up whole buffers that it+ * didn't really need to. We can't postpone advancing RedoRecPtr because+ * XLogInserts that happen while we are dumping buffers must assume that+ * their buffer changes are not included in the checkpoint.+ */+ {+ /* use volatile pointer to prevent code rearrangement */+ volatile XLogCtlData *xlogctl = XLogCtl;+ + SpinLockAcquire(&xlogctl->info_lck);+ RedoRecPtr = xlogctl->Insert.RedoRecPtr = redo;+ SpinLockRelease(&xlogctl->info_lck);+ }+ + return redo;+ }+
/*
* Flush all data in shared memory to disk, and fsync
*
***************
*** 6073,6101 ****
}
}
/*
! * OK, force data out to disk
*/
! CheckPointGuts(checkPoint->redo, CHECKPOINT_IMMEDIATE);
/*
! * Update pg_control so that any subsequent crash will restart from this
! * checkpoint. Note: ReadRecPtr gives the XLOG address of the checkpoint
! * record itself.
*/
ControlFile->prevCheckPoint = ControlFile->checkPoint;
! ControlFile->checkPoint = ReadRecPtr;
! ControlFile->checkPointCopy = *checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
ereport((recoveryLogRestartpoints ? LOG : DEBUG2),
(errmsg("recovery restart point at %X/%X",
! checkPoint->redo.xlogid, checkPoint->redo.xrecoff)));
if (recoveryLastXTime)
ereport((recoveryLogRestartpoints ? LOG : DEBUG2),
! (errmsg("last completed transaction was at log time %s",
! timestamptz_to_str(recoveryLastXTime))));
}
/*
--- 6323,6391 ----
}
}
+ RequestRestartPoint(ReadRecPtr, checkPoint, reachedSafeStartPoint);+ }+ + /*+ * As of 8.4, RestartPoints are always created by the bgwriter+ * once we have reachedSafeStartPoint. We use bgwriter's shared memory+ * area wherever we call it from, to keep better code structure.+ */+ void+ CreateRestartPoint(const XLogRecPtr ReadPtr, const CheckPoint *restartPoint, int flags)+ {+ if (recoveryLogRestartpoints)+ {+ /*+ * Prepare to accumulate statistics.+ */+ + MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));+ CheckpointStats.ckpt_start_t = GetCurrentTimestamp();+ + LogCheckpointStart(CHECKPOINT_RESTARTPOINT | flags);+ }+
/*
! * Acquire CheckpointLock to ensure only one restartpoint happens at a time.
! * We rely on this lock to ensure that the startup process doesn't exit
! * Recovery while we are half way through a restartpoint.
*/
! LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
!
! CheckPointGuts(restartPoint->redo, CHECKPOINT_RESTARTPOINT | flags);
/*
! * Update pg_control, using current time
*/
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
ControlFile->prevCheckPoint = ControlFile->checkPoint;
! ControlFile->checkPoint = ReadPtr;
! ControlFile->checkPointCopy = *restartPoint;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
+ LWLockRelease(ControlFileLock);+ + /*+ * Currently, there is no need to truncate pg_subtrans during recovery.+ * If we did do that, we will need to have called StartupSUBTRANS()+ * already and then TruncateSUBTRANS() would go here.+ */+ + /* All real work is done, but log before releasing lock. */+ if (recoveryLogRestartpoints)+ LogCheckpointEnd(CHECKPOINT_RESTARTPOINT);
ereport((recoveryLogRestartpoints ? LOG : DEBUG2),
(errmsg("recovery restart point at %X/%X",
! restartPoint->redo.xlogid, restartPoint->redo.xrecoff)));
!
if (recoveryLastXTime)
ereport((recoveryLogRestartpoints ? LOG : DEBUG2),
! (errmsg("last completed transaction was at log time %s",
! timestamptz_to_str(recoveryLastXTime))));
!
! LWLockRelease(CheckpointLock);
}
/*
***************
*** 6160,6166 ****
}
/*
! * XLOG resource manager's routines
*/
void
xlog_redo(XLogRecPtr lsn, XLogRecord *record)
--- 6450,6512 ----
}
/*
! * exitRecovery()
! *
! * Exit recovery state and write a XLOG_RECOVERY_END record. This is the
! * only record type that can record a change of timelineID. We assume
! * caller has already set ThisTimeLineID, if appropriate.
! */
! static void
! exitRecovery(void)
! {
! XLogRecData rdata;
!
! rdata.buffer = InvalidBuffer;
! rdata.data = (char *) (&ThisTimeLineID);
! rdata.len = sizeof(TimeLineID);
! rdata.next = NULL;
!
! /*
! * If a restartpoint is in progress, we will not be able to successfully
! * acquire CheckpointLock. If bgwriter is still in progress then send
! * a second signal to nudge bgwriter to go faster so we can avoid delay.
! * Then wait for lock, so we know the restartpoint has completed. We do
! * this because we don't want to interrupt the restartpoint half way
! * through, which might leave us in a mess and we want to be robust. We're
! * going to checkpoint soon anyway, so not it's not wasted effort.
! */
! if (LWLockConditionalAcquire(CheckpointLock, LW_EXCLUSIVE))
! LWLockRelease(CheckpointLock);
! else
! {
! RequestRestartPointCompletion();
! ereport(LOG,
! (errmsg("startup process waiting for restartpoint to complete")));
! LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
! LWLockRelease(CheckpointLock);
! }
!
! /*
! * This is the only type of WAL message that can be inserted during
! * recovery. This ensures that we don't allow others to get access
! * until after we have changed state.
! */
! (void) XLogInsert(RM_XLOG_ID, XLOG_RECOVERY_END, &rdata);
!
! /*
! * We don't XLogFlush() here otherwise we'll end up zeroing the WAL
! * file ourselves. So just let bgwriter's forthcoming checkpoint do
! * that for us.
! */
!
! InRecovery = false;
! }
!
! /*
! * XLOG resource manager's routines.
! *
! * Definitions of message info are in include/catalog/pg_control.h,
! * though not all messages relate to control file processing.
*/
void
xlog_redo(XLogRecPtr lsn, XLogRecord *record)
***************
*** 6195,6215 ****
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
/*
! * TLI may change in a shutdown checkpoint, but it shouldn't decrease
*/
! if (checkPoint.ThisTimeLineID != ThisTimeLineID)
{
! if (checkPoint.ThisTimeLineID < ThisTimeLineID ||
!list_member_int(expectedTLIs,
! (int) checkPoint.ThisTimeLineID))
ereport(PANIC,
! (errmsg("unexpected timeline ID %u (after %u) in checkpoint record",
! checkPoint.ThisTimeLineID, ThisTimeLineID)));
/* Following WAL records should be run with new TLI */
! ThisTimeLineID = checkPoint.ThisTimeLineID;
}
- - RecoveryRestartPoint(&checkPoint);
}
else if (info == XLOG_CHECKPOINT_ONLINE)
{
--- 6541,6578 ----
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
/*
! * TLI no longer changes at shutdown checkpoint, since as of 8.4,
! * shutdown checkpoints only occur at shutdown. Much less confusing.
*/
!
! RecoveryRestartPoint(&checkPoint);
! }
! else if (info == XLOG_RECOVERY_END)
! {
! TimeLineID tli;
!
! memcpy(&tli, XLogRecGetData(record), sizeof(TimeLineID));
!
! /*
! * TLI may change when recovery ends, but it shouldn't decrease.
! *
! * This is the only WAL record that can tell us to change timelineID
! * while we process WAL records.
! *
! * We can *choose* to stop recovery at any point, generating a
! * new timelineID which is recorded using this record type.
! */
! if (tli != ThisTimeLineID)
{
! if (tli < ThisTimeLineID ||
!list_member_int(expectedTLIs,
! (int) tli))
ereport(PANIC,
! (errmsg("unexpected timeline ID %u (after %u) at recovery end record",
! tli, ThisTimeLineID)));
/* Following WAL records should be run with new TLI */
! ThisTimeLineID = tli;
}
}
else if (info == XLOG_CHECKPOINT_ONLINE)
{
***************
*** 6232,6238 ****
ControlFile->checkPointCopy.nextXidEpoch = checkPoint.nextXidEpoch;
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
! /* TLI should not change in an on-line checkpoint */
if (checkPoint.ThisTimeLineID != ThisTimeLineID)
ereport(PANIC,
(errmsg("unexpected timeline ID %u (should be %u) in checkpoint record",
--- 6595,6601 ----
ControlFile->checkPointCopy.nextXidEpoch = checkPoint.nextXidEpoch;
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
! /* TLI must not change at a checkpoint */
if (checkPoint.ThisTimeLineID != ThisTimeLineID)
ereport(PANIC,
(errmsg("unexpected timeline ID %u (should be %u) in checkpoint record",
Index: src/backend/postmaster/bgwriter.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/postmaster/bgwriter.c,v
retrieving revision 1.52
diff -c -r1.52 bgwriter.c
*** src/backend/postmaster/bgwriter.c 30 Sep 2008 10:52:13 -0000 1.52
--- src/backend/postmaster/bgwriter.c 8 Oct 2008 13:05:35 -0000
***************
*** 49,54 ****
--- 49,55 ----
#include <unistd.h>
#include "access/xlog_internal.h"
+ #include "catalog/pg_control.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
***************
*** 129,134 ****
--- 130,142 ----
int ckpt_flags; /* checkpoint flags, as defined in xlog.h */
+ /* + * When the Startup process wants bgwriter to perform a restartpoint, it + * sets these fields so that we can update the control file afterwards.+ */+ XLogRecPtr ReadPtr; /* Requested log pointer */+ CheckPoint restartPoint; /* restartPoint data for ControlFile */+
uint32 num_backend_writes; /* counts non-bgwriter buffer writes */
int num_requests; /* current # of requests */
***************
*** 165,171 ****
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
! static XLogRecPtr ckpt_start_recptr;
static double ckpt_cached_elapsed;
static pg_time_t last_checkpoint_time;
--- 173,179 ----
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
! static XLogRecPtr ckpt_start_recptr; /* not used if IsRecoveryProcessingMode */
static double ckpt_cached_elapsed;
static pg_time_t last_checkpoint_time;
***************
*** 197,202 ****
--- 205,211 ----
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
+ bool BgWriterRecoveryMode;
BgWriterShmem->bgwriter_pid = MyProcPid;
am_bg_writer = true;
***************
*** 355,370 ****
*/
PG_SETMASK(&UnBlockSig);
/*
* Loop forever
*/
for (;;)
{
- bool do_checkpoint = false;- int flags = 0;- pg_time_t now;- int elapsed_secs;-
/*
* Emergency bailout if postmaster has died. This is to avoid the
* necessity for manual cleanup of all postmaster children.
--- 364,380 ----
*/
PG_SETMASK(&UnBlockSig);
+ BgWriterRecoveryMode = IsRecoveryProcessingMode();+ + if (BgWriterRecoveryMode)+ elog(DEBUG1, "bgwriter starting during recovery, pid = %u", + BgWriterShmem->bgwriter_pid);+
/*
* Loop forever
*/
for (;;)
{
/*
* Emergency bailout if postmaster has died. This is to avoid the
* necessity for manual cleanup of all postmaster children.
***************
*** 382,499 ****
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
}
- if (checkpoint_requested)- {- checkpoint_requested = false;- do_checkpoint = true;- BgWriterStats.m_requested_checkpoints++;- }- if (shutdown_requested)- {- /*- * From here on, elog(ERROR) should end with exit(1), not send- * control back to the sigsetjmp block above- */- ExitOnAnyError = true;- /* Close down the database */- ShutdownXLOG(0, 0);- /* Normal exit from the bgwriter is here */- proc_exit(0); /* done */- }- - /*- * Force a checkpoint if too much time has elapsed since the last one.- * Note that we count a timed checkpoint in stats only when this- * occurs without an external request, but we set the CAUSE_TIME flag- * bit even if there is also an external request.- */- now = (pg_time_t) time(NULL);- elapsed_secs = now - last_checkpoint_time;- if (elapsed_secs >= CheckPointTimeout)- {- if (!do_checkpoint)- BgWriterStats.m_timed_checkpoints++;- do_checkpoint = true;- flags |= CHECKPOINT_CAUSE_TIME;- }- - /*- * Do a checkpoint if requested, otherwise do one cycle of- * dirty-buffer writing.- */- if (do_checkpoint)- {- /* use volatile pointer to prevent code rearrangement */- volatile BgWriterShmemStruct *bgs = BgWriterShmem;- - /*- * Atomically fetch the request flags to figure out what kind of a- * checkpoint we should perform, and increase the started-counter- * to acknowledge that we've started a new checkpoint.- */- SpinLockAcquire(&bgs->ckpt_lck);- flags |= bgs->ckpt_flags;- bgs->ckpt_flags = 0;- bgs->ckpt_started++;- SpinLockRelease(&bgs->ckpt_lck);
! /*
! * We will warn if (a) too soon since last checkpoint (whatever
! * caused it) and (b) somebody set the CHECKPOINT_CAUSE_XLOG flag
! * since the last checkpoint start. Note in particular that this
! * implementation will not generate warnings caused by
! * CheckPointTimeout < CheckPointWarning.
! */
! if ((flags & CHECKPOINT_CAUSE_XLOG) &&
! elapsed_secs < CheckPointWarning)
! ereport(LOG,
! (errmsg("checkpoints are occurring too frequently (%d seconds apart)",
! elapsed_secs),
! errhint("Consider increasing the configuration parameter \"checkpoint_segments\".")));
/*
! * Initialize bgwriter-private variables used during checkpoint.
*/
! ckpt_active = true;
! ckpt_start_recptr = GetInsertRecPtr();
! ckpt_start_time = now;
! ckpt_cached_elapsed = 0;
/*
! * Do the checkpoint.
*/
! CreateCheckPoint(flags);
! /*
! * After any checkpoint, close all smgr files. This is so we
! * won't hang onto smgr references to deleted files indefinitely.
! */
! smgrcloseall();
! /*
! * Indicate checkpoint completion to any waiting backends.
! */
! SpinLockAcquire(&bgs->ckpt_lck);
! bgs->ckpt_done = bgs->ckpt_started;
! SpinLockRelease(&bgs->ckpt_lck);
!
! ckpt_active = false;
!
! /*
! * Note we record the checkpoint start time not end time as
! * last_checkpoint_time. This is so that time-driven checkpoints
! * happen at a predictable spacing.
! */
! last_checkpoint_time = now;
}
- else- BgBufferSync();- - /* Check for archive_timeout and switch xlog files if necessary. */- CheckArchiveTimeout();- - /* Nap for the configured time. */- BgWriterNap();
}
}
--- 392,595 ----
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
}
! if (BgWriterRecoveryMode)
! {
! if (shutdown_requested)
! {
! /*
! * From here on, elog(ERROR) should end with exit(1), not send
! * control back to the sigsetjmp block above
! */
! ExitOnAnyError = true;
! /* Normal exit from the bgwriter is here */
! proc_exit(0); /* done */
! }
!
! if (!IsRecoveryProcessingMode())
! {
! elog(DEBUG2, "bgwriter changing from recovery to normal mode");
!
! InitXLOGAccess();
! BgWriterRecoveryMode = false;
!
! /*
! * Start time-driven events from now
! */
! last_checkpoint_time = last_xlog_switch_time = (pg_time_t) time(NULL);
!
! /*
! * Notice that we do *not* act on a checkpoint_requested
! * state at this point. We have changed mode, so we wish to
! * perform a checkpoint not a restartpoint.
! */
! continue;
! }
!
! if (checkpoint_requested)
! {
! XLogRecPtr ReadPtr;
! CheckPoint restartPoint;
!
! checkpoint_requested = false;
!
! /*
! * Initialize bgwriter-private variables used during checkpoint.
! */
! ckpt_active = true;
! ckpt_start_time = (pg_time_t) time(NULL);
! ckpt_cached_elapsed = 0;
!
! /*
! * Get the requested values from shared memory that the
! * Startup process has put there for us.
! */
! SpinLockAcquire(&BgWriterShmem->ckpt_lck);
! ReadPtr = BgWriterShmem->ReadPtr;
! memcpy(&restartPoint, &BgWriterShmem->restartPoint, sizeof(CheckPoint));
! SpinLockRelease(&BgWriterShmem->ckpt_lck);
!
! /* Use smoothed writes, until interrupted if ever */
! CreateRestartPoint(ReadPtr, &restartPoint, 0);
!
! /*
! * After any checkpoint, close all smgr files. This is so we
! * won't hang onto smgr references to deleted files indefinitely.
! */
! smgrcloseall();
!
! ckpt_active = false;
! checkpoint_requested = false;
! }
! else
! {
! /* Clean buffers dirtied by recovery */
! BgBufferSync();
!
! /* Nap for the configured time. */
! BgWriterNap();
! }
! }
! else /* Normal processing */
! {
! bool do_checkpoint = false;
! int flags = 0;
! pg_time_t now;
! int elapsed_secs;
!
! if (checkpoint_requested)
! {
! checkpoint_requested = false;
! do_checkpoint = true;
! BgWriterStats.m_requested_checkpoints++;
! }
! if (shutdown_requested)
! {
! /*
! * From here on, elog(ERROR) should end with exit(1), not send
! * control back to the sigsetjmp block above
! */
! ExitOnAnyError = true;
! /* Close down the database */
! ShutdownXLOG(0, 0);
! /* Normal exit from the bgwriter is here */
! proc_exit(0); /* done */
! }
/*
! * Force a checkpoint if too much time has elapsed since the last one.
! * Note that we count a timed checkpoint in stats only when this
! * occurs without an external request, but we set the CAUSE_TIME flag
! * bit even if there is also an external request.
*/
! now = (pg_time_t) time(NULL);
! elapsed_secs = now - last_checkpoint_time;
! if (elapsed_secs >= CheckPointTimeout)
! {
! if (!do_checkpoint)
! BgWriterStats.m_timed_checkpoints++;
! do_checkpoint = true;
! flags |= CHECKPOINT_CAUSE_TIME;
! }
/*
! * Do a checkpoint if requested, otherwise do one cycle of
! * dirty-buffer writing.
*/
! if (do_checkpoint)
! {
! /* use volatile pointer to prevent code rearrangement */
! volatile BgWriterShmemStruct *bgs = BgWriterShmem;
!
! /*
! * Atomically fetch the request flags to figure out what kind of a
! * checkpoint we should perform, and increase the started-counter
! * to acknowledge that we've started a new checkpoint.
! */
! SpinLockAcquire(&bgs->ckpt_lck);
! flags |= bgs->ckpt_flags;
! bgs->ckpt_flags = 0;
! bgs->ckpt_started++;
! SpinLockRelease(&bgs->ckpt_lck);
!
! /*
! * We will warn if (a) too soon since last checkpoint (whatever
! * caused it) and (b) somebody set the CHECKPOINT_CAUSE_XLOG flag
! * since the last checkpoint start. Note in particular that this
! * implementation will not generate warnings caused by
! * CheckPointTimeout < CheckPointWarning.
! */
! if ((flags & CHECKPOINT_CAUSE_XLOG) &&
! elapsed_secs < CheckPointWarning)
! ereport(LOG,
! (errmsg("checkpoints are occurring too frequently (%d seconds apart)",
! elapsed_secs),
! errhint("Consider increasing the configuration parameter \"checkpoint_segments\".")));
!
! /*
! * Initialize bgwriter-private variables used during checkpoint.
! */
! ckpt_active = true;
! ckpt_start_recptr = GetInsertRecPtr();
! ckpt_start_time = now;
! ckpt_cached_elapsed = 0;
!
! /*
! * Do the checkpoint.
! */
! CreateCheckPoint(flags);
!
! /*
! * After any checkpoint, close all smgr files. This is so we
! * won't hang onto smgr references to deleted files indefinitely.
! */
! smgrcloseall();
!
! /*
! * Indicate checkpoint completion to any waiting backends.
! */
! SpinLockAcquire(&bgs->ckpt_lck);
! bgs->ckpt_done = bgs->ckpt_started;
! SpinLockRelease(&bgs->ckpt_lck);
!
! ckpt_active = false;
!
! /*
! * Note we record the checkpoint start time not end time as
! * last_checkpoint_time. This is so that time-driven checkpoints
! * happen at a predictable spacing.
! */
! last_checkpoint_time = now;
! }
! else
! BgBufferSync();
! /* Check for archive_timeout and switch xlog files if necessary. */
! CheckArchiveTimeout();
! /* Nap for the configured time. */
! BgWriterNap();
}
}
}
***************
*** 586,592 ****
(ckpt_active ? ImmediateCheckpointRequested() : checkpoint_requested))
break;
pg_usleep(1000000L);
! AbsorbFsyncRequests();
udelay -= 1000000L;
}
--- 682,689 ----
(ckpt_active ? ImmediateCheckpointRequested() : checkpoint_requested))
break;
pg_usleep(1000000L);
! if (!IsRecoveryProcessingMode())
! AbsorbFsyncRequests();
udelay -= 1000000L;
}
***************
*** 640,645 ****
--- 737,755 ----
if (!am_bg_writer)
return;
+ /* Perform minimal duties during recovery and skip wait if requested */+ if (IsRecoveryProcessingMode())+ {+ BgBufferSync();+ + if (!shutdown_requested &&+ !checkpoint_requested &&+ IsCheckpointOnSchedule(progress))+ BgWriterNap();+ + return;+ }+
/*
* Perform the usual bgwriter duties and take a nap, unless we're behind
* schedule, in which case we just try to catch up as quickly as possible.
***************
*** 714,729 ****
* However, it's good enough for our purposes, we're only calculating an
* estimate anyway.
*/
! recptr = GetInsertRecPtr();
! elapsed_xlogs =
! (((double) (int32) (recptr.xlogid - ckpt_start_recptr.xlogid)) * XLogSegsPerFile +
! ((double) recptr.xrecoff - (double) ckpt_start_recptr.xrecoff) / XLogSegSize) /
! CheckPointSegments;
!
! if (progress < elapsed_xlogs)
{
! ckpt_cached_elapsed = elapsed_xlogs;
! return false;
}
/*
--- 824,842 ----
* However, it's good enough for our purposes, we're only calculating an
* estimate anyway.
*/
! if (!IsRecoveryProcessingMode())
{
! recptr = GetInsertRecPtr();
! elapsed_xlogs =
! (((double) (int32) (recptr.xlogid - ckpt_start_recptr.xlogid)) * XLogSegsPerFile +
! ((double) recptr.xrecoff - (double) ckpt_start_recptr.xrecoff) / XLogSegSize) /
! CheckPointSegments;
!
! if (progress < elapsed_xlogs)
! {
! ckpt_cached_elapsed = elapsed_xlogs;
! return false;
! }
}
/*
***************
*** 965,970 ****
--- 1078,1154 ----
}
/*
+ * Always runs in Startup process (see xlog.c)+ */+ void+ RequestRestartPoint(const XLogRecPtr ReadPtr, const CheckPoint *restartPoint, bool sendToBGWriter)+ {+ /*+ * Should we just do it ourselves?+ */+ if (!IsPostmasterEnvironment || !sendToBGWriter)+ {+ CreateRestartPoint(ReadPtr, restartPoint, CHECKPOINT_IMMEDIATE);+ return;+ }+ + /*+ * Push requested values into shared memory, then signal to request restartpoint.+ */+ if (BgWriterShmem->bgwriter_pid == 0)+ elog(LOG, "could not request restartpoint because bgwriter not running");+ + #ifdef NOT_USED+ elog(LOG, "tli = %u nextXidEpoch = %u nextXid = %u nextOid = %u",+ restartPoint->ThisTimeLineID,+ restartPoint->nextXidEpoch,+ restartPoint->nextXid,+ restartPoint->nextOid);+ #endif+ + SpinLockAcquire(&BgWriterShmem->ckpt_lck);+ BgWriterShmem->ReadPtr = ReadPtr;+ memcpy(&BgWriterShmem->restartPoint, restartPoint, sizeof(CheckPoint));+ SpinLockRelease(&BgWriterShmem->ckpt_lck);+ + if (kill(BgWriterShmem->bgwriter_pid, SIGINT) != 0)+ elog(LOG, "could not signal for restartpoint: %m"); + }+ + /* + * Sends another checkpoint request signal to bgwriter, which causes it+ * to avoid smoothed writes and continue processing as if it had been+ * called with CHECKPOINT_IMMEDIATE. This is used at the end of recovery.+ */+ void+ RequestRestartPointCompletion(void)+ {+ if (BgWriterShmem->bgwriter_pid != 0 &&+ kill(BgWriterShmem->bgwriter_pid, SIGINT) != 0)+ elog(LOG, "could not signal for restartpoint immediate: %m");+ }+ + XLogRecPtr+ GetRedoLocationForArchiveCheckpoint(void)+ {+ XLogRecPtr redo;+ + SpinLockAcquire(&BgWriterShmem->ckpt_lck);+ redo = BgWriterShmem->ReadPtr;+ SpinLockRelease(&BgWriterShmem->ckpt_lck);+ + return redo;+ }+ + void+ SetRedoLocationForArchiveCheckpoint(XLogRecPtr redo)+ {+ SpinLockAcquire(&BgWriterShmem->ckpt_lck);+ BgWriterShmem->ReadPtr = redo;+ SpinLockRelease(&BgWriterShmem->ckpt_lck);+ }+ + /*
* ForwardFsyncRequest
* Forward a file-fsync request from a backend to the bgwriter
*
Index: src/backend/postmaster/postmaster.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/postmaster/postmaster.c,v
retrieving revision 1.565
diff -c -r1.565 postmaster.c
*** src/backend/postmaster/postmaster.c 23 Sep 2008 20:35:38 -0000 1.565
--- src/backend/postmaster/postmaster.c 8 Oct 2008 12:27:33 -0000
***************
*** 254,259 ****
--- 254,264 ----
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_RECOVERY, /* consistent recovery mode; state only+ * entered for archive and streaming recovery,+ * and only after the point where the + * all data is in consistent state.+ */
PM_RUN, /* normal "database is alive" state */
PM_WAIT_BACKUP, /* waiting for online backup mode to end */
PM_WAIT_BACKENDS, /* waiting for live backends to exit */
***************
*** 1302,1308 ****
* state that prevents it, start one. It doesn't matter if this
* fails, we'll just try again later.
*/
! if (BgWriterPID == 0 && pmState == PM_RUN)
BgWriterPID = StartBackgroundWriter();
/*
--- 1307,1313 ----
* state that prevents it, start one. It doesn't matter if this
* fails, we'll just try again later.
*/
! if (BgWriterPID == 0 && (pmState == PM_RUN || pmState == PM_RECOVERY))
BgWriterPID = StartBackgroundWriter();
/*
***************
*** 2116,2122 ****
if (pid == StartupPID)
{
StartupPID = 0;
! Assert(pmState == PM_STARTUP);
/* FATAL exit of startup is treated as catastrophic */
if (!EXIT_STATUS_0(exitstatus))
--- 2121,2127 ----
if (pid == StartupPID)
{
StartupPID = 0;
! Assert(pmState == PM_STARTUP || pmState == PM_RECOVERY);
/* FATAL exit of startup is treated as catastrophic */
if (!EXIT_STATUS_0(exitstatus))
***************
*** 2157,2167 ****
load_role();
/*
! * Crank up the background writer. It doesn't matter if this
! * fails, we'll just try again later.
*/
! Assert(BgWriterPID == 0);
! BgWriterPID = StartBackgroundWriter();
/*
* Likewise, start other special children as needed. In a restart
--- 2162,2172 ----
load_role();
/*
! * Check whether we need to start background writer, if not
! * already running.
*/
! if (BgWriterPID == 0)
! BgWriterPID = StartBackgroundWriter();
/*
* Likewise, start other special children as needed. In a restart
***************
*** 3845,3850 ****
--- 3850,3900 ----
PG_SETMASK(&BlockSig);
+ if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_START))+ {+ Assert(pmState == PM_STARTUP);+ + /*+ * Go to shutdown mode if a shutdown request was pending.+ */+ if (Shutdown > NoShutdown)+ {+ pmState = PM_WAIT_BACKENDS;+ /* PostmasterStateMachine logic does the rest */+ }+ else+ {+ /*+ * Startup process has entered recovery+ */+ pmState = PM_RECOVERY;+ + /*+ * Load the flat authorization file into postmaster's cache. The+ * startup process won't have recomputed this from the database yet,+ * so we it may change following recovery. + */+ load_role();+ + /*+ * Crank up the background writer. It doesn't matter if this+ * fails, we'll just try again later.+ */+ Assert(BgWriterPID == 0);+ BgWriterPID = StartBackgroundWriter();+ + /*+ * Likewise, start other special children as needed.+ */+ Assert(PgStatPID == 0);+ PgStatPID = pgstat_start();+ + /* XXX at this point we could accept read-only connections */+ ereport(DEBUG1,+ (errmsg("database system is in consistent recovery mode")));+ }+ }+
if (CheckPostmasterSignal(PMSIGNAL_PASSWORD_CHANGE))
{
/*
Index: src/backend/storage/buffer/README
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/storage/buffer/README,v
retrieving revision 1.14
diff -c -r1.14 README
*** src/backend/storage/buffer/README 21 Mar 2008 13:23:28 -0000 1.14
--- src/backend/storage/buffer/README 8 Oct 2008 12:27:33 -0000
***************
*** 264,266 ****
--- 264,275 ----
This ensures that the page image transferred to disk is reasonably consistent.
We might miss a hint-bit update or two but that isn't a problem, for the same
reasons mentioned under buffer access rules.
+ + As of 8.4, background writer starts during recovery mode when there is+ some form of potentially extended recovery to perform. It performs an+ identical service to normal processing, except that checkpoints it+ writes are technically restartpoints. Flushing outstanding WAL for dirty+ buffers is also skipped, though there shouldn't ever be new WAL entries+ at that time in any case. We could choose to start background writer+ immediately but we hold off until we can prove the database is in a + consistent state so that postmaster has a single, clean state change.
Index: src/bin/pg_controldata/pg_controldata.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/bin/pg_controldata/pg_controldata.c,v
retrieving revision 1.41
diff -c -r1.41 pg_controldata.c
*** src/bin/pg_controldata/pg_controldata.c 24 Sep 2008 08:59:42 -0000 1.41
--- src/bin/pg_controldata/pg_controldata.c 8 Oct 2008 12:27:33 -0000
***************
*** 197,202 ****
--- 197,205 ----
printf(_("Minimum recovery ending location: %X/%X\n"),
ControlFile.minRecoveryPoint.xlogid,
ControlFile.minRecoveryPoint.xrecoff);
+ printf(_("Minimum safe starting location: %X/%X\n"),+ ControlFile.minSafeStartPoint.xlogid,+ ControlFile.minSafeStartPoint.xrecoff);
printf(_("Maximum data alignment: %u\n"),
ControlFile.maxAlign);
/* we don't print floatFormat since can't say much useful about it */
Index: src/bin/pg_resetxlog/pg_resetxlog.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/bin/pg_resetxlog/pg_resetxlog.c,v
retrieving revision 1.68
diff -c -r1.68 pg_resetxlog.c
*** src/bin/pg_resetxlog/pg_resetxlog.c 24 Sep 2008 09:00:44 -0000 1.68
--- src/bin/pg_resetxlog/pg_resetxlog.c 8 Oct 2008 12:27:33 -0000
***************
*** 595,600 ****
--- 595,602 ----
ControlFile.prevCheckPoint.xrecoff = 0;
ControlFile.minRecoveryPoint.xlogid = 0;
ControlFile.minRecoveryPoint.xrecoff = 0;
+ ControlFile.minSafeStartPoint.xlogid = 0;+ ControlFile.minSafeStartPoint.xrecoff = 0;
/* Now we can force the recorded xlog seg size to the right thing. */
ControlFile.xlog_seg_size = XLogSegSize;
Index: src/include/access/xlog.h
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/access/xlog.h,v
retrieving revision 1.88
diff -c -r1.88 xlog.h
*** src/include/access/xlog.h 12 May 2008 08:35:05 -0000 1.88
--- src/include/access/xlog.h 8 Oct 2008 12:27:33 -0000
***************
*** 133,139 ****
} XLogRecData;
extern TimeLineID ThisTimeLineID; /* current TLI */
! extern bool InRecovery;
extern XLogRecPtr XactLastRecEnd;
/* these variables are GUC parameters related to XLOG */
--- 133,148 ----
} XLogRecData;
extern TimeLineID ThisTimeLineID; /* current TLI */
!
! /*
! * Prior to 8.4, all activity during recovery were carried out by Startup
! * process. This local variable continues to be used in many parts of the
! * code to indicate actions taken by RecoveryManagers. Other processes who
! * potentially perform work during recovery should check
! * IsRecoveryProcessingMode(), see XLogCtl notes in xlog.c
! */
! extern bool InRecovery;
!
extern XLogRecPtr XactLastRecEnd;
/* these variables are GUC parameters related to XLOG */
***************
*** 166,171 ****
--- 175,181 ----
/* These indicate the cause of a checkpoint request */
#define CHECKPOINT_CAUSE_XLOG 0x0010 /* XLOG consumption */
#define CHECKPOINT_CAUSE_TIME 0x0020 /* Elapsed time */
+ #define CHECKPOINT_RESTARTPOINT 0x0040 /* Restartpoint during recovery */
/* Checkpoint statistics */
typedef struct CheckpointStatsData
***************
*** 197,202 ****
--- 207,214 ----
extern void xlog_redo(XLogRecPtr lsn, XLogRecord *record);
extern void xlog_desc(StringInfo buf, uint8 xl_info, char *rec);
+ extern bool IsRecoveryProcessingMode(void);+
extern void UpdateControlFile(void);
extern Size XLOGShmemSize(void);
extern void XLOGShmemInit(void);
Index: src/include/access/xlog_internal.h
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/access/xlog_internal.h,v
retrieving revision 1.24
diff -c -r1.24 xlog_internal.h
*** src/include/access/xlog_internal.h 11 Aug 2008 11:05:11 -0000 1.24
--- src/include/access/xlog_internal.h 8 Oct 2008 12:27:33 -0000
***************
*** 17,22 ****
--- 17,23 ----
#define XLOG_INTERNAL_H
#include "access/xlog.h"
+ #include "catalog/pg_control.h"
#include "fmgr.h"
#include "pgtime.h"
#include "storage/block.h"
***************
*** 245,250 ****
--- 246,254 ----
extern pg_time_t GetLastSegSwitchTime(void);
extern XLogRecPtr RequestXLogSwitch(void);
+ extern void CreateRestartPoint(const XLogRecPtr ReadPtr, + const CheckPoint *restartPoint, int flags);+
/*
* These aren't in xlog.h because I'd rather not include fmgr.h there.
*/
Index: src/include/catalog/pg_control.h
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/catalog/pg_control.h,v
retrieving revision 1.42
diff -c -r1.42 pg_control.h
*** src/include/catalog/pg_control.h 23 Sep 2008 09:20:39 -0000 1.42
--- src/include/catalog/pg_control.h 8 Oct 2008 12:27:33 -0000
***************
*** 46,52 ****
#define XLOG_NOOP 0x20
#define XLOG_NEXTOID 0x30
#define XLOG_SWITCH 0x40
!
/* System status indicator */
typedef enum DBState
--- 46,52 ----
#define XLOG_NOOP 0x20
#define XLOG_NEXTOID 0x30
#define XLOG_SWITCH 0x40
! #define XLOG_RECOVERY_END 0x50
/* System status indicator */
typedef enum DBState
***************
*** 102,107 ****
--- 102,108 ----
CheckPoint checkPointCopy; /* copy of last check point record */
XLogRecPtr minRecoveryPoint; /* must replay xlog to here */
+ XLogRecPtr minSafeStartPoint; /* safe point after recovery crashes */
/*
* This data is used to check for hardware-architecture compatibility of
Index: src/include/postmaster/bgwriter.h
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/postmaster/bgwriter.h,v
retrieving revision 1.12
diff -c -r1.12 bgwriter.h
*** src/include/postmaster/bgwriter.h 11 Aug 2008 11:05:11 -0000 1.12
--- src/include/postmaster/bgwriter.h 8 Oct 2008 12:27:33 -0000
***************
*** 12,17 ****
--- 12,18 ----
#ifndef _BGWRITER_H
#define _BGWRITER_H
+ #include "catalog/pg_control.h"
#include "storage/block.h"
#include "storage/relfilenode.h"
***************
*** 25,30 ****
--- 26,36 ----
extern void BackgroundWriterMain(void);
extern void RequestCheckpoint(int flags);
+ extern void RequestRestartPoint(const XLogRecPtr ReadPtr, const CheckPoint *restartPoint, bool sendToBGWriter);+ extern void RequestRestartPointCompletion(void);+ extern XLogRecPtr GetRedoLocationForArchiveCheckpoint(void);+ extern void SetRedoLocationForArchiveCheckpoint(XLogRecPtr redo);+
extern void CheckpointWriteDelay(int flags, double progress);
extern bool ForwardFsyncRequest(RelFileNode rnode, ForkNumber forknum,
Index: src/include/storage/pmsignal.h
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/storage/pmsignal.h,v
retrieving revision 1.20
diff -c -r1.20 pmsignal.h
*** src/include/storage/pmsignal.h 19 Jun 2008 21:32:56 -0000 1.20
--- src/include/storage/pmsignal.h 8 Oct 2008 12:27:33 -0000
***************
*** 22,27 ****
--- 22,28 ----
*/
typedef enum
{
+ PMSIGNAL_RECOVERY_START, /* move to PM_RECOVERY state */
PMSIGNAL_PASSWORD_CHANGE, /* pg_auth file has changed */
PMSIGNAL_WAKEN_ARCHIVER, /* send a NOTIFY signal to xlog archiver */
PMSIGNAL_ROTATE_LOGFILE, /* send SIGUSR1 to syslogger to rotate logfile */
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-patches@postgresql.org
Cc: simon@2ndQuadrant.com, heikki.linnakangas@enterprisedb.com, pgsql-hackers@postgresql.org
Subject: Re: [PATCHES] Infrastructure changes for recovery (v8)
In-Reply-To: <1223472898.4747.310.camel@ebony.2ndQuadrant>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox