public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 2/4] demote: support demoting instance from production to standby
42+ messages / 3 participants
[nested] [flat]
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
Procedure and Architecture:
* demote can be triggered using "pg_ctl [-m {fast|smart}] demote"
or using the SQL admin function
"pg_demote(fast bool, wait bool, wait_seconds int)"
* on sigusr1, postmaster check for demote signal files
"demote" or "demote_fast" in PGDATA
* if in production and a demote signal file is found, the demote
process starts by setting PM_DEMOTING
* set DB_DEMOTING state in controlfile
* PM_DEMOTING waits for all backends to be in read only
* every idle backends set LocalXLogInsert=0 immediatly to
forbid new writes
* in fast mode, every backend holding a xid is terminated
* in smart mode, wait for running xact to finish
* once all backends are read only, set PM_SHUTDOWN to create a
shutdown checkpoint
* during the shutdown chechpoint, ShutdownXLOG now takes a boolean
arg to handle demote differently than a normal shutdown
* the shutdown checkpoint set the cluster state as DB_DEMOTING in
the controlfile
* the checkpointer then exits to have a fresh restart for code
simplicity
* Postmaster sets PM_STARTUP on checkpointer exit status
* the startup process is then started from PostmasterStateMachine()
and try to handle subsystems init correctly during demote
* the demote procedure keeps some sub-processes alive:
stat collector, bgwriter and optionally archiver and wal senders
* at the end of demote, send USR1 to signal the backends and wal
senders to set their environment as in recovery and cascading
Discuss/Todo:
* add doc
* code reviewing
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 95 +++++++
src/backend/access/transam/xlog.c | 315 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 +++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 7 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 799 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0e..fda085631f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1565,6 +1565,101 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = ProcGlobal->allPgXact[gxact->pgprocno].xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 25a9f78690..4aaa138b1b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextFullXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextFullXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
-
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,55 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ LocalRecoveryInProgress = true;
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8640,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8779,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8808,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8821,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8876,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8922,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9090,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9110,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9130,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9144,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 15f92b66c6..d20b8d8530 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5b5fc97c72..8004770d8c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -275,12 +275,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -324,6 +325,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -414,10 +416,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2305,6 +2311,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2436,10 +2447,11 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test;
+ * we expect bgworker_should_start_now() decided whether the DB state
+ * allows them.
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
@@ -2452,6 +2464,8 @@ canAcceptConnections(int backend_type)
{
if (pmState == PM_WAIT_BACKUP)
result = CAC_WAITBACKUP; /* allow superusers only */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
@@ -3108,7 +3122,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3802,6 +3827,25 @@ PostmasterStateMachine(void)
pmState = PM_WAIT_BACKENDS;
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
if (pmState == PM_WAIT_READONLY)
{
/*
@@ -3995,6 +4039,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5205,8 +5263,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5214,8 +5276,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5226,6 +5287,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5236,11 +5298,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5251,7 +5317,17 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
}
@@ -5342,6 +5418,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5439,6 +5606,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGXACT *xact = &ProcGlobal->allPgXact[i];
+ if (TransactionIdIsValid(xact->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5904,6 +6091,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6652,3 +6840,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b448533564..0ccc32f4ce 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -191,6 +191,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * TODO demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79b..52f85cd1b3 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4371,6 +4371,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..6bd1e1e1d0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyPgXact->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df74..c144cc35d3 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e538..f529f8c7bd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 179ebaa104..a9e27f009e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,12 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_DEMOTE,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_TOOMANY,
CAC_WAITBACKUP
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index fdabf42721..d3b08163a2 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -574,6 +574,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/Mp45B_GpB5m14pibp/TRwlo
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 2/4] demote: support demoting instance from production to standby
@ 2020-04-10 16:01 Jehan-Guillaume de Rorthais <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Jehan-Guillaume de Rorthais @ 2020-04-10 16:01 UTC (permalink / raw)
This patch adds the ability for an in production read write
instance to step back as a read only standby. Two different
modes are supported: fast or smart. Fast demote mode cancel
transactions holding a xid to demote as fast as possible.
Smart demote mode wait for existing transactions holding a
xid to finish and forbid new ones.
The demote process is triggered by creating a "demote" or
"demote_fast" trigger file then signaling postmaster with
SIGUSR1. This patch adds "pg_ctl [-m {fast|smart}] demote"
and an SQL admin function will be added in a separate commit.
The demote procedure starts be setting the postmaster state
machine to PM_DEMOTING which ends when all backends are
read-only and set LocalXLogInsert=0 to forbid new writes.
When PM_DEMOTING finish, the state is set to PM_SHUTDOWN
to create a shutdown checkpoint and exit the checkpointer.
We could have kept the checkpointer around with some more
effort but there's no good reason worthing the additionnal
code complexity.
ShutdownXLOG now takes a boolean arg to handle demote
differently than a normal shutdown. This allows the
checkpointer to leave walsenders alive to demote faster.
Moreover, this might be useful in futur for eg.
implementing a controlled switchover over the replication
protocol.
The checkpointer set the cluster state as DB_DEMOTING so
the Startup process can detect the demote procedure in
StartupXLOG() and handle subsystems accordingly.
The startup process is started on checkpointer exit. The
postmaster state machine is then switched to PM_STARTUP.
Some sub-processes are kept alive during the demote procedure:
the stat collector, bgwriter and optionally archiver and wal
senders.
At the end of demote, USR1 is sent to all backends and wal
senders to set their environment as in recovery and cascading.
Discuss/Todo:
* add doc
* do not handle backup in progress during demote
* investigate snapshots shmem needs/init during recovery compare to
production
* cancel running checkpoint during demote
* replace with a END_OF_PRODUCTION xlog record?
---
src/backend/access/transam/twophase.c | 98 ++++++++
src/backend/access/transam/xlog.c | 320 ++++++++++++++++--------
src/backend/postmaster/checkpointer.c | 28 +++
src/backend/postmaster/pgstat.c | 3 +
src/backend/postmaster/postmaster.c | 235 ++++++++++++++++-
src/backend/storage/ipc/procarray.c | 2 +
src/backend/storage/ipc/procsignal.c | 30 +++
src/backend/storage/lmgr/lock.c | 12 +
src/backend/tcop/postgres.c | 44 ++++
src/backend/utils/init/globals.c | 1 +
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_ctl/pg_ctl.c | 117 +++++++++
src/include/access/twophase.h | 1 +
src/include/access/xlog.h | 23 +-
src/include/catalog/pg_control.h | 1 +
src/include/libpq/libpq-be.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 1 +
src/include/postmaster/bgwriter.h | 1 +
src/include/storage/lock.h | 2 +
src/include/storage/procsignal.h | 4 +
src/include/tcop/tcopprot.h | 2 +
src/include/utils/pidfile.h | 1 +
23 files changed, 802 insertions(+), 129 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e3..31deb487ed 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1557,6 +1557,104 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
pfree(buf);
}
+/*
+ * ShutdownPreparedTransactions: clean prepared from sheared memory
+ *
+ * This is called during the demote process to clean the shared memory
+ * before the startup process load everything back in correctly
+ * for the standby mode.
+ *
+ * Note: this function assue all prepared transaction have been
+ * written to disk. In consequence, it must be called AFTER the demote
+ * shutdown checkpoint.
+ *
+ * FIXME demote: pay attention to the previous note when removing shutdown
+ * checkpoint from the demote procedure.
+ */
+void
+ShutdownPreparedTransactions(void)
+{
+ int i;
+
+ for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+ {
+ GlobalTransaction gxact;
+ PGPROC *proc;
+ TransactionId xid;
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+ TransactionId latestXid;
+ TransactionId *children;
+
+ gxact = TwoPhaseState->prepXacts[i];
+ proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ xid = gxact->xid;
+
+ /* Read and validate 2PC state data */
+ Assert(gxact->ondisk);
+ buf = ReadTwoPhaseFile(xid, false);
+
+ /*
+ * Disassemble the header area
+ */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader))
+ + MAXALIGN(hdr->gidlen);
+ children = (TransactionId *) bufptr;
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId))
+ + MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->nabortrels * sizeof(RelFileNode))
+ + MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+ /* compute latestXid among all children */
+ latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+
+ /* remove dummy proc associated to the gaxt */
+ ProcArrayRemove(proc, latestXid);
+
+ /*
+ * This lock is probably not needed during the demote process
+ * as all backends are already gone.
+ */
+ LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+ /* cleanup locks */
+ for (;;)
+ {
+ TwoPhaseRecordOnDisk *record = (TwoPhaseRecordOnDisk *) bufptr;
+
+ Assert(record->rmid <= TWOPHASE_RM_MAX_ID);
+ if (record->rmid == TWOPHASE_RM_END_ID)
+ break;
+
+ bufptr += MAXALIGN(sizeof(TwoPhaseRecordOnDisk));
+
+ if (record->rmid == TWOPHASE_RM_LOCK_ID)
+ lock_twophase_shutdown(xid, record->info,
+ (void *) bufptr, record->len);
+
+ bufptr += MAXALIGN(record->len);
+ }
+
+ /* and put it back in the freelist */
+ gxact->next = TwoPhaseState->freeGXacts;
+ TwoPhaseState->freeGXacts = gxact;
+
+ /*
+ * Release the lock as all callbacks are called and shared memory cleanup
+ * is done.
+ */
+ LWLockRelease(TwoPhaseStateLock);
+
+ pfree(buf);
+ }
+
+ TwoPhaseState->numPrepXacts -= i;
+ Assert(TwoPhaseState->numPrepXacts == 0);
+}
+
/*
* Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c0d79f192c..abc88ec241 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6298,6 +6298,11 @@ CheckRequiredParameterValues(void)
/*
* This must be called ONCE during postmaster or standalone-backend startup
*/
+/*
+ * FIXME demote: part of the code here assume there's no other active
+ * processes before signal PMSIGNAL_RECOVERY_STARTED is sent.
+ */
+
void
StartupXLOG(void)
{
@@ -6321,6 +6326,7 @@ StartupXLOG(void)
XLogPageReadPrivate private;
bool promoted = false;
struct stat st;
+ bool is_demoting = false;
/*
* We should have an aux process resource owner to use, and we should not
@@ -6385,6 +6391,25 @@ StartupXLOG(void)
str_time(ControlFile->time))));
break;
+ case DB_DEMOTING:
+ ereport(LOG,
+ (errmsg("database system was demoted at %s",
+ str_time(ControlFile->time))));
+ is_demoting = true;
+ bgwriterLaunched = true;
+ InArchiveRecovery = true;
+ StandbyMode = true;
+
+ /*
+ * previous state was RECOVERY_STATE_DONE. We need to
+ * reinit it to something else so RecoveryInProgress()
+ * doesn't return false.
+ */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ break;
+
default:
ereport(FATAL,
(errmsg("control file contains invalid database cluster state")));
@@ -6418,7 +6443,8 @@ StartupXLOG(void)
* persisted. To avoid that, fsync the entire data directory.
*/
if (ControlFile->state != DB_SHUTDOWNED &&
- ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY &&
+ ControlFile->state != DB_DEMOTING)
{
RemoveTempXlogFiles();
SyncDataDirectory();
@@ -6674,7 +6700,8 @@ StartupXLOG(void)
(errmsg("could not locate a valid checkpoint record")));
}
memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint));
- wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN);
+ wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN) &&
+ !is_demoting;
}
/*
@@ -6736,9 +6763,9 @@ StartupXLOG(void)
LastRec = RecPtr = checkPointLoc;
ereport(DEBUG1,
- (errmsg_internal("redo record is at %X/%X; shutdown %s",
+ (errmsg_internal("redo record is at %X/%X; %s checkpoint",
(uint32) (checkPoint.redo >> 32), (uint32) checkPoint.redo,
- wasShutdown ? "true" : "false")));
+ wasShutdown ? "shutdown" : is_demoting? "demote": "")));
ereport(DEBUG1,
(errmsg_internal("next transaction ID: " UINT64_FORMAT "; next OID: %u",
U64FromFullTransactionId(checkPoint.nextXid),
@@ -6772,47 +6799,7 @@ StartupXLOG(void)
checkPoint.newestCommitTsXid);
XLogCtl->ckptFullXid = checkPoint.nextXid;
- /*
- * Initialize replication slots, before there's a chance to remove
- * required resources.
- */
- StartupReplicationSlots();
-
- /*
- * Startup logical state, needs to be setup now so we have proper data
- * during crash recovery.
- */
- StartupReorderBuffer();
- /*
- * Startup MultiXact. We need to do this early to be able to replay
- * truncations.
- */
- StartupMultiXact();
-
- /*
- * Ditto for commit timestamps. Activate the facility if the setting is
- * enabled in the control file, as there should be no tracking of commit
- * timestamps done when the setting was disabled. This facility can be
- * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
- */
- if (ControlFile->track_commit_timestamp)
- StartupCommitTs();
-
- /*
- * Recover knowledge about replay progress of known replication partners.
- */
- StartupReplicationOrigin();
-
- /*
- * Initialize unlogged LSN. On a clean shutdown, it's restored from the
- * control file. On recovery, all unlogged relations are blown away, so
- * the unlogged LSN counter can be reset too.
- */
- if (ControlFile->state == DB_SHUTDOWNED)
- XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
- else
- XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
/*
* We must replay WAL entries using the same TimeLineID they were created
@@ -6821,19 +6808,64 @@ StartupXLOG(void)
*/
ThisTimeLineID = checkPoint.ThisTimeLineID;
- /*
- * Copy any missing timeline history files between 'now' and the recovery
- * target timeline from archive to pg_wal. While we don't need those files
- * ourselves - the history file of the recovery target timeline covers all
- * the previous timelines in the history too - a cascading standby server
- * might be interested in them. Or, if you archive the WAL from this
- * server to a different archive than the primary, it'd be good for all the
- * history files to get archived there after failover, so that you can use
- * one of the old timelines as a PITR target. Timeline history files are
- * small, so it's better to copy them unnecessarily than not copy them and
- * regret later.
- */
- restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ if (!is_demoting)
+ {
+ /*
+ * Initialize replication slots, before there's a chance to remove
+ * required resources.
+ */
+ StartupReplicationSlots();
+
+ /*
+ * Startup logical state, needs to be setup now so we have proper data
+ * during crash recovery.
+ */
+ StartupReorderBuffer();
+
+ /*
+ * Startup MultiXact. We need to do this early to be able to replay
+ * truncations.
+ */
+ StartupMultiXact();
+
+ /*
+ * Ditto for commit timestamps. Activate the facility if the setting is
+ * enabled in the control file, as there should be no tracking of commit
+ * timestamps done when the setting was disabled. This facility can be
+ * started or stopped when replaying a XLOG_PARAMETER_CHANGE record.
+ */
+ if (ControlFile->track_commit_timestamp)
+ StartupCommitTs();
+
+ /*
+ * Recover knowledge about replay progress of known replication partners.
+ */
+ StartupReplicationOrigin();
+
+ /*
+ * Initialize unlogged LSN. On a clean shutdown, it's restored from the
+ * control file. On recovery, all unlogged relations are blown away, so
+ * the unlogged LSN counter can be reset too.
+ */
+ if (ControlFile->state == DB_SHUTDOWNED)
+ XLogCtl->unloggedLSN = ControlFile->unloggedLSN;
+ else
+ XLogCtl->unloggedLSN = FirstNormalUnloggedLSN;
+
+ /*
+ * Copy any missing timeline history files between 'now' and the recovery
+ * target timeline from archive to pg_wal. While we don't need those files
+ * ourselves - the history file of the recovery target timeline covers all
+ * the previous timelines in the history too - a cascading standby server
+ * might be interested in them. Or, if you archive the WAL from this
+ * server to a different archive than the master, it'd be good for all the
+ * history files to get archived there after failover, so that you can use
+ * one of the old timelines as a PITR target. Timeline history files are
+ * small, so it's better to copy them unnecessarily than not copy them and
+ * regret later.
+ */
+ restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
+ }
/*
* Before running in recovery, scan pg_twophase and fill in its status to
@@ -6888,11 +6920,25 @@ StartupXLOG(void)
dbstate_at_startup = ControlFile->state;
if (InArchiveRecovery)
{
- ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ if (is_demoting)
+ {
+ /*
+ * Avoid concurrent access to the ControlFile datas
+ * during demotion.
+ */
+ LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
+ LWLockRelease(ControlFileLock);
+ }
+ else
+ {
+ ControlFile->state = DB_IN_ARCHIVE_RECOVERY;
- SpinLockAcquire(&XLogCtl->info_lck);
- XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
- SpinLockRelease(&XLogCtl->info_lck);
+ /* This is already set if demoting */
+ SpinLockAcquire(&XLogCtl->info_lck);
+ XLogCtl->SharedRecoveryState = RECOVERY_STATE_ARCHIVE;
+ SpinLockRelease(&XLogCtl->info_lck);
+ }
}
else
{
@@ -6982,7 +7028,8 @@ StartupXLOG(void)
/*
* Reset pgstat data, because it may be invalid after recovery.
*/
- pgstat_reset_all();
+ if (!is_demoting)
+ pgstat_reset_all();
/*
* If there was a backup label file, it's done its job and the info
@@ -7044,7 +7091,7 @@ StartupXLOG(void)
InitRecoveryTransactionEnvironment();
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
else
oldestActiveXID = checkPoint.oldestActiveXid;
@@ -7057,6 +7104,11 @@ StartupXLOG(void)
* Startup commit log and subtrans only. MultiXact and commit
* timestamp have already been started up and other SLRUs are not
* maintained during recovery and need not be started yet.
+ *
+ * Starting up commit log is technicaly not needed during demote
+ * as the in-memory data did not move. However, this is a
+ * lightweight initialization and this might seem expected as
+ * pure symmetry as ShutdownCLOG() is called during ShutdownXLog().
*/
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
@@ -7067,7 +7119,7 @@ StartupXLOG(void)
* empty running-xacts record and use that here and now. Recover
* additional standby state for prepared transactions.
*/
- if (wasShutdown)
+ if (wasShutdown || is_demoting)
{
RunningTransactionsData running;
TransactionId latestCompletedXid;
@@ -7938,6 +7990,7 @@ StartupXLOG(void)
SpinLockAcquire(&XLogCtl->info_lck);
XLogCtl->SharedRecoveryState = RECOVERY_STATE_DONE;
+ XLogCtl->SharedHotStandbyActive = false;
SpinLockRelease(&XLogCtl->info_lck);
UpdateControlFile();
@@ -8056,6 +8109,23 @@ CheckRecoveryConsistency(void)
}
}
+/*
+ * Initialize the local TimeLineID
+ */
+bool
+SetLocalRecoveryInProgress(void)
+{
+ /*
+ * use volatile pointer to make sure we make a fresh read of the
+ * shared variable.
+ */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+
+ return LocalRecoveryInProgress;
+}
+
/*
* Is the system still in recovery?
*
@@ -8077,13 +8147,7 @@ RecoveryInProgress(void)
return false;
else
{
- /*
- * use volatile pointer to make sure we make a fresh read of the
- * shared variable.
- */
- volatile XLogCtlData *xlogctl = XLogCtl;
-
- LocalRecoveryInProgress = (xlogctl->SharedRecoveryState != RECOVERY_STATE_DONE);
+ SetLocalRecoveryInProgress();
/*
* Initialize TimeLineID and RedoRecPtr when we discover that recovery
@@ -8503,6 +8567,8 @@ GetLastSegSwitchData(XLogRecPtr *lastSwitchLSN)
void
ShutdownXLOG(int code, Datum arg)
{
+ bool is_demoting = DatumGetBool(arg);
+
/*
* We should have an aux process resource owner to use, and we should not
* be in a transaction that's installed some other resowner.
@@ -8512,35 +8578,60 @@ ShutdownXLOG(int code, Datum arg)
CurrentResourceOwner == AuxProcessResourceOwner);
CurrentResourceOwner = AuxProcessResourceOwner;
- /* Don't be chatty in standalone mode */
- ereport(IsPostmasterEnvironment ? LOG : NOTICE,
- (errmsg("shutting down")));
-
- /*
- * Signal walsenders to move to stopping state.
- */
- WalSndInitStopping();
-
- /*
- * Wait for WAL senders to be in stopping state. This prevents commands
- * from writing new WAL.
- */
- WalSndWaitStopping();
+ if (is_demoting)
+ {
+ /*
+ * In contrast with normal shutdown, we keep wal senders alive
+ * during demote. First, this allows the demote to complete faster.
+ * Second, we might need them in futur to implement a controlled
+ * switchover over the replication protocol.
+ */
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("demoting")));
- if (RecoveryInProgress())
- CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * FIXME demote: avoiding checkpoint?
+ * A checkpoint is probably running during a demote action. If
+ * we don't want to wait for the checkpoint during the demote,
+ * we might need to cancel it as it will not be able to write
+ * to the WAL after the demote.
+ */
+ CreateCheckPoint(CHECKPOINT_IS_DEMOTE | CHECKPOINT_IMMEDIATE);
+ ShutdownPreparedTransactions();
+ }
else
{
+ /* Don't be chatty in standalone mode */
+ ereport(IsPostmasterEnvironment ? LOG : NOTICE,
+ (errmsg("shutting down")));
+
/*
- * If archiving is enabled, rotate the last XLOG file so that all the
- * remaining records are archived (postmaster wakes up the archiver
- * process one more time at the end of shutdown). The checkpoint
- * record will go to the next XLOG file and won't be archived (yet).
+ * Signal walsenders to move to stopping state.
*/
- if (XLogArchivingActive() && XLogArchiveCommandSet())
- RequestXLogSwitch(false);
+ WalSndInitStopping();
+
+ /*
+ * Wait for WAL senders to be in stopping state. This prevents commands
+ * from writing new WAL.
+ */
+ WalSndWaitStopping();
+
+ if (RecoveryInProgress())
+ CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ else
+ {
+ /*
+ * If archiving is enabled, rotate the last XLOG file so that all the
+ * remaining records are archived (postmaster wakes up the archiver
+ * process one more time at the end of shutdown). The checkpoint
+ * record will go to the next XLOG file and won't be archived (yet).
+ */
+ if (XLogArchivingActive() && XLogArchiveCommandSet())
+ RequestXLogSwitch(false);
- CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ }
}
ShutdownCLOG();
ShutdownCommitTs();
@@ -8554,9 +8645,10 @@ ShutdownXLOG(int code, Datum arg)
static void
LogCheckpointStart(int flags, bool restartpoint)
{
- elog(LOG, "%s starting:%s%s%s%s%s%s%s%s",
+ elog(LOG, "%s starting:%s%s%s%s%s%s%s%s%s",
restartpoint ? "restartpoint" : "checkpoint",
(flags & CHECKPOINT_IS_SHUTDOWN) ? " shutdown" : "",
+ (flags & CHECKPOINT_IS_DEMOTE) ? " demote" : "",
(flags & CHECKPOINT_END_OF_RECOVERY) ? " end-of-recovery" : "",
(flags & CHECKPOINT_IMMEDIATE) ? " immediate" : "",
(flags & CHECKPOINT_FORCE) ? " force" : "",
@@ -8692,6 +8784,7 @@ UpdateCheckPointDistanceEstimate(uint64 nbytes)
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
+ * CHECKPOINT_IS_DEMOTE: checkpoint is for demote.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
@@ -8720,6 +8813,7 @@ void
CreateCheckPoint(int flags)
{
bool shutdown;
+ bool demote;
CheckPoint checkPoint;
XLogRecPtr recptr;
XLogSegNo _logSegNo;
@@ -8732,14 +8826,21 @@ CreateCheckPoint(int flags)
int nvxids;
/*
- * An end-of-recovery checkpoint is really a shutdown checkpoint, just
- * issued at a different time.
+ * An end-of-recovery or demote checkpoint is really a shutdown checkpoint,
+ * just issued at a different time.
*/
- if (flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY))
+ if (flags & (CHECKPOINT_IS_SHUTDOWN |
+ CHECKPOINT_IS_DEMOTE |
+ CHECKPOINT_END_OF_RECOVERY))
shutdown = true;
else
shutdown = false;
+ if (flags & CHECKPOINT_IS_DEMOTE)
+ demote = true;
+ else
+ demote = false;
+
/* sanity check */
if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
elog(ERROR, "can't create a checkpoint during recovery");
@@ -8780,7 +8881,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNING;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
@@ -8826,7 +8927,7 @@ CreateCheckPoint(int flags)
* avoid inserting duplicate checkpoints when the system is idle.
*/
if ((flags & (CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY |
- CHECKPOINT_FORCE)) == 0)
+ CHECKPOINT_IS_DEMOTE | CHECKPOINT_FORCE)) == 0)
{
if (last_important_lsn == ControlFile->checkPoint)
{
@@ -8994,8 +9095,8 @@ CreateCheckPoint(int flags)
* allows us to reconstruct the state of running transactions during
* archive recovery, if required. Skip, if this info disabled.
*
- * If we are shutting down, or Startup process is completing crash
- * recovery we don't need to write running xact data.
+ * If we are shutting down, demoting or Startup process is completing
+ * crash recovery we don't need to write running xact data.
*/
if (!shutdown && XLogStandbyInfoActive())
LogStandbySnapshot();
@@ -9014,11 +9115,11 @@ CreateCheckPoint(int flags)
XLogFlush(recptr);
/*
- * We mustn't write any new WAL after a shutdown checkpoint, or it will be
- * overwritten at next startup. No-one should even try, this just allows
- * sanity-checking. In the case of an end-of-recovery checkpoint, we want
- * to just temporarily disable writing until the system has exited
- * recovery.
+ * We mustn't write any new WAL after a shutdown or demote checkpoint, or
+ * it will be overwritten at next startup. No-one should even try, this
+ * just allows sanity-checking. In the case of an end-of-recovery
+ * checkpoint, we want to just temporarily disable writing until the system
+ * has exited recovery.
*/
if (shutdown)
{
@@ -9034,7 +9135,8 @@ CreateCheckPoint(int flags)
*/
if (shutdown && checkPoint.redo != ProcLastRecPtr)
ereport(PANIC,
- (errmsg("concurrent write-ahead log activity while database system is shutting down")));
+ (errmsg("concurrent write-ahead log activity while database system is %s",
+ demote? "demoting":"shutting down")));
/*
* Remember the prior checkpoint's redo ptr for
@@ -9047,7 +9149,7 @@ CreateCheckPoint(int flags)
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
if (shutdown)
- ControlFile->state = DB_SHUTDOWNED;
+ ControlFile->state = demote? DB_DEMOTING:DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
ControlFile->time = (pg_time_t) time(NULL);
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..58473a61fd 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -151,6 +151,7 @@ double CheckPointCompletionTarget = 0.5;
* Private state
*/
static bool ckpt_active = false;
+static volatile sig_atomic_t demoteRequestPending = false;
/* these values are valid when ckpt_active is true: */
static pg_time_t ckpt_start_time;
@@ -552,6 +553,21 @@ HandleCheckpointerInterrupts(void)
*/
UpdateSharedMemoryConfig();
}
+ if (demoteRequestPending)
+ {
+ demoteRequestPending = false;
+ /* Close down the database */
+ ShutdownXLOG(0, BoolGetDatum(true));
+ /*
+ * Exit checkpointer. We could keep it around during demotion, but
+ * exiting here has multiple benefices:
+ * - to create a fresh process with clean local vars
+ * (eg. LocalRecoveryInProgress)
+ * - to signal postmaster the demote shutdown checkpoint is done
+ * and keep going with next steps of the demotion
+ */
+ proc_exit(0);
+ }
if (ShutdownRequestPending)
{
/*
@@ -680,6 +696,7 @@ CheckpointWriteDelay(int flags, double progress)
* in which case we just try to catch up as quickly as possible.
*/
if (!(flags & CHECKPOINT_IMMEDIATE) &&
+ !demoteRequestPending &&
!ShutdownRequestPending &&
!ImmediateCheckpointRequested() &&
IsCheckpointOnSchedule(progress))
@@ -812,6 +829,17 @@ IsCheckpointOnSchedule(double progress)
* --------------------------------
*/
+/* SIGUSR1: set flag to demote */
+void
+ReqCheckpointDemoteHandler(SIGNAL_ARGS)
+{
+ int save_errno = errno;
+
+ demoteRequestPending = true;
+
+ errno = save_errno;
+}
+
/* SIGINT: set flag to run a normal checkpoint right away */
static void
ReqCheckpointHandler(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1..e75d33d335 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3854,6 +3854,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_PROMOTE:
event_name = "Promote";
break;
+ case WAIT_EVENT_DEMOTE:
+ event_name = "Demote";
+ break;
case WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT:
event_name = "RecoveryConflictSnapshot";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..231febaf2f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -277,12 +277,13 @@ static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
#define ImmediateShutdown 3
static int Shutdown = NoShutdown;
+static bool DemoteSignal = false; /* true on demote request */
static bool FatalError = false; /* T if recovering from backend crash */
/*
- * We use a simple state machine to control startup, shutdown, and
- * crash recovery (which is rather like shutdown followed by startup).
+ * We use a simple state machine to control startup, shutdown, demote and
+ * crash recovery (both are rather like shutdown followed by startup).
*
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
@@ -325,6 +326,7 @@ typedef enum
{
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
+ PM_DEMOTING, /* waiting for idle or RO backends for demote */
PM_RECOVERY, /* in archive recovery mode */
PM_HOT_STANDBY, /* in hot standby mode */
PM_RUN, /* normal "database is alive" state */
@@ -429,10 +431,14 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+static void RemoveDemoteSignalFiles(void);
+static bool CheckDemoteSignal(void);
+
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
+static int CountXacts(void);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
@@ -2319,6 +2325,11 @@ retry1:
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
break;
+ case CAC_DEMOTE:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is demoting")));
+ break;
case CAC_SHUTDOWN:
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
@@ -2450,16 +2461,19 @@ canAcceptConnections(int backend_type)
CAC_state result = CAC_OK;
/*
- * Can't start backends when in startup/shutdown/inconsistent recovery
- * state. We treat autovac workers the same as user backends for this
- * purpose. However, bgworkers are excluded from this test; we expect
- * bgworker_should_start_now() decided whether the DB state allows them.
+ * Can't start backends when in startup/demote/shutdown/inconsistent
+ * recovery state. We treat autovac workers the same as user backends
+ * for this purpose. However, bgworkers are excluded from this test; we
+ * expect bgworker_should_start_now() decided whether the DB state allows
+ * them.
*/
if (pmState != PM_RUN && pmState != PM_HOT_STANDBY &&
backend_type != BACKEND_TYPE_BGWORKER)
{
if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
+ else if (DemoteSignal)
+ return CAC_DEMOTE; /* demote is pending */
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
@@ -3091,7 +3105,18 @@ reaper(SIGNAL_ARGS)
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
- if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
+ if (EXIT_STATUS_0(exitstatus) &&
+ DemoteSignal &&
+ pmState == PM_SHUTDOWN)
+ {
+ /*
+ * The checkpointer exit signals the demote shutdown checkpoint
+ * is done. The startup recovery mode can be started from there.
+ */
+ ereport(DEBUG1,
+ (errmsg_internal("checkpointer shutdown for demote")));
+ }
+ else if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
{
/*
* OK, we saw normal exit of the checkpointer after it's been
@@ -3799,6 +3824,25 @@ PostmasterStateMachine(void)
}
}
+ if (pmState == PM_DEMOTING)
+ {
+ int numXacts = CountXacts();
+
+ /*
+ * PM_DEMOTING state ends when we have no active transactions
+ * and all backends set LocalXLogInsertAllowed=0
+ */
+ if (numXacts == 0)
+ {
+ ereport(LOG, (errmsg("all backends in read only")));
+
+ SendProcSignal(CheckpointerPID, PROCSIG_CHECKPOINTER_DEMOTING, InvalidBackendId);
+ pmState = PM_SHUTDOWN;
+ }
+ else
+ ereport(LOG, (errmsg("waiting for %d transactions to finish", numXacts)));
+ }
+
/*
* If we're ready to do so, signal child processes to shut down. (This
* isn't a persistent state, but treating it as a distinct pmState allows
@@ -4002,6 +4046,20 @@ PostmasterStateMachine(void)
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
ExitPostmaster(1);
+
+ /* Demoting: start the Startup Process */
+ if (DemoteSignal && pmState == PM_SHUTDOWN && CheckpointerPID == 0)
+ {
+ /* stop archiver process if not required during standby */
+ if (!XLogArchivingAlways() && PgArchPID != 0)
+ signal_child(PgArchPID, SIGQUIT);
+
+ StartupPID = StartupDataBase();
+ Assert(StartupPID != 0);
+ StartupStatus = STARTUP_RUNNING;
+ pmState = PM_STARTUP;
+ }
+
/*
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
@@ -5212,8 +5270,12 @@ sigusr1_handler(SIGNAL_ARGS)
* Crank up the background tasks. It doesn't matter if this fails,
* we'll just try again later.
*/
+ if (!DemoteSignal)
+ Assert(PgArchPID == 0);
+
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
+
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
@@ -5221,8 +5283,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
+ if (PgArchPID == 0 && XLogArchivingAlways())
PgArchPID = pgarch_start();
/*
@@ -5233,6 +5294,7 @@ sigusr1_handler(SIGNAL_ARGS)
if (!EnableHotStandby)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+ DemoteSignal = false;
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
@@ -5243,11 +5305,15 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
+ dlist_iter iter;
+
/*
* Likewise, start other special children as needed.
*/
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ if (!DemoteSignal)
+ Assert(PgStatPID == 0);
+ if(PgStatPID == 0)
+ PgStatPID = pgstat_start();
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5258,8 +5324,18 @@ sigusr1_handler(SIGNAL_ARGS)
sd_notify(0, "READY=1");
#endif
+ if (DemoteSignal)
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (!bp->dead_end && bp->bkend_type & (BACKEND_TYPE_NORMAL|BACKEND_TYPE_WALSND))
+ SendProcSignal(bp->pid, PROCSIG_DEMOTED, InvalidBackendId);
+ }
+
pmState = PM_HOT_STANDBY;
connsAllowed = ALLOW_ALL_CONNS;
+ DemoteSignal = false;
/* Some workers may be scheduled to start now */
StartWorkerNeeded = true;
@@ -5351,6 +5427,97 @@ sigusr1_handler(SIGNAL_ARGS)
signal_child(StartupPID, SIGUSR2);
}
+ if (CheckDemoteSignal() && pmState != PM_RUN )
+ {
+ DemoteSignal = false;
+ RemoveDemoteSignalFiles();
+ ereport(LOG,
+ (errmsg("ignoring demote signal because already in standby mode")));
+ }
+ /* received demote signal */
+ else if (CheckDemoteSignal())
+ {
+ FILE *standby_file;
+ dlist_iter iter;
+ bool fast_demote;
+ struct stat stat_buf;
+
+ fast_demote = (stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0);
+
+ DemoteSignal = true;
+ RemoveDemoteSignalFiles();
+
+ /* create the standby signal file */
+ standby_file = AllocateFile(STANDBY_SIGNAL_FILE, "w");
+ if (!standby_file)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ goto out;
+ }
+
+ if (FreeFile(standby_file))
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ STANDBY_SIGNAL_FILE)));
+ unlink(STANDBY_SIGNAL_FILE);
+ goto out;
+ }
+
+ if (fast_demote == 0)
+ {
+ /* smart demote */
+ ereport(LOG, (errmsg("received smart demote request")));
+
+ }
+ else
+ {
+ /* fast demote */
+ ereport(LOG, (errmsg("received fast demote request")));
+ }
+
+ SignalSomeChildren(SIGTERM,
+ BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
+
+ /* and the autovac launcher too */
+ if (AutoVacPID != 0)
+ signal_child(AutoVacPID, SIGTERM);
+ /* and the bgwriter too */
+ if (BgWriterPID != 0)
+ signal_child(BgWriterPID, SIGTERM);
+ /* and the walwriter too */
+ if (WalWriterPID != 0)
+ signal_child(WalWriterPID, SIGTERM);
+
+ dlist_foreach(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->dead_end)
+ continue;
+ /*
+ * Assign bkend_type for any recently announced WAL Sender
+ * processes.
+ */
+ if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
+ ! IsPostmasterChildWalSender(bp->child_slot))
+ SendProcSignal(bp->pid,
+ (fast_demote?PROCSIG_DEMOTING_FAST:PROCSIG_DEMOTING),
+ InvalidBackendId);
+ }
+
+ pmState = PM_DEMOTING;
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_DEMOTING);
+ }
+
+out:
+
#ifdef WIN32
PG_SETMASK(&UnBlockSig);
#endif
@@ -5448,6 +5615,26 @@ CountChildren(int target)
}
+/*
+ * Count up the number of active transactions
+ */
+static int
+CountXacts(void)
+{
+ int i;
+ int cnt = 0;
+
+ for (i = 0; i < ProcGlobal->allProcCount; ++i)
+ {
+ PGPROC *proc = &ProcGlobal->allProcs[i];
+ if (TransactionIdIsValid(proc->xid))
+ cnt++;
+ }
+
+ return cnt;
+}
+
+
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5912,6 +6099,7 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_STOP_BACKENDS:
+ case PM_DEMOTING:
break;
case PM_RUN:
@@ -6660,3 +6848,28 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+/*
+ * Remove the files signaling a demote request.
+ */
+static void
+RemoveDemoteSignalFiles(void)
+{
+ unlink(DEMOTE_SIGNAL_FILE);
+ unlink(DEMOTE_FAST_SIGNAL_FILE);
+}
+
+/*
+ * Check if a demote request appeared.
+ */
+static bool
+CheckDemoteSignal(void)
+{
+ struct stat stat_buf;
+
+ if (stat(DEMOTE_SIGNAL_FILE, &stat_buf) == 0 ||
+ stat(DEMOTE_FAST_SIGNAL_FILE, &stat_buf) == 0)
+ return true;
+
+ return false;
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857..83d7e05944 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -349,6 +349,8 @@ ProcArrayShmemSize(void)
size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
/*
+ * FIXME demote: check safe hotStandby related init and snapshot mech.
+ *
* During Hot Standby processing we have a data structure called
* KnownAssignedXids, created in shared memory. Local data structures are
* also created in various backends during GetSnapshotData(),
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ec..ac14c662d3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -28,6 +28,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "postmaster/bgwriter.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -585,6 +586,35 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ /* signal checkpoint process to ignite a demote procedure */
+ if (CheckProcSignal(PROCSIG_CHECKPOINTER_DEMOTING))
+ ReqCheckpointDemoteHandler(PROCSIG_CHECKPOINTER_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 as soon as their active xact
+ * finished
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING))
+ ReqDemoteHandler(PROCSIG_DEMOTING);
+
+ /*
+ * ask backends to enter in read only by setting
+ * LocalXLogInsertAllowed = 0 if they are idle, or
+ * interrupt their current xact and terminate.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTING_FAST))
+ ReqDemoteHandler(PROCSIG_DEMOTING_FAST);
+
+ /*
+ * demote complete. Ask beckends to rely on
+ * recovery status for LocalXLogInsertAllowed by
+ * setting it to -1.
+ * WAL sender set am_cascading.
+ */
+ if (CheckProcSignal(PROCSIG_DEMOTED))
+ ReqDemotedHandler(PROCSIG_DEMOTED);
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f455..5843db0991 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -4370,6 +4370,18 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
lock_twophase_postcommit(xid, info, recdata, len);
}
+/*
+ * 2PC shutdown from lock table.
+ *
+ * This is actually just the same as the COMMIT case.
+ */
+void
+lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ lock_twophase_postcommit(xid, info, recdata, len);
+}
+
/*
* VirtualXactLockTableInsert
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..b44e6f5876 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -67,6 +67,7 @@
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
#include "storage/ipc.h"
+#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinval.h"
@@ -3211,6 +3212,42 @@ ProcessInterrupts(void)
HandleParallelMessages();
}
+/* SIGUSR1: set flag to demote */
+void
+ReqDemoteHandler(ProcSignalReason reason)
+{
+ if (MyBackendType != B_BACKEND)
+ return;
+
+ if (TransactionIdIsValid(MyProc->xid))
+ {
+ if (reason == PROCSIG_DEMOTING_FAST)
+ {
+ InterruptPending = true;
+ ProcDiePending = true;
+ SetLatch(MyLatch);
+ }
+ else
+ DemotePending = true;
+ }
+ else
+ LocalSetXLogInsertNotAllowed();
+}
+
+/* SIGUSR1: reset LocalRecoveryInProgress */
+void
+ReqDemotedHandler(ProcSignalReason reason)
+{
+ ereport(LOG,
+ (errmsg("received demote complete signal")));
+
+ SetLocalRecoveryInProgress();
+ LocalSetXLogInsertCheckRecovery();
+
+ if (MyBackendType == B_WAL_SENDER)
+ am_cascading_walsender = true;
+}
+
/*
* IA64-specific code to fetch the AR.BSP register for stack depth checks.
@@ -4224,6 +4261,12 @@ PostgresMain(int argc, char *argv[],
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
+ if (DemotePending) {
+ LocalSetXLogInsertNotAllowed();
+ DemotePending = false;
+ SendPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE);
+ }
+
/*
* Also process incoming notifies, if any. This is mostly to
* ensure stable behavior in tests: if any notifies were
@@ -4285,6 +4328,7 @@ PostgresMain(int argc, char *argv[],
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
+ SetLocalRecoveryInProgress();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..021f6af434 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t DemotePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f70..9ef133f79a 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -57,6 +57,8 @@ dbState(DBState state)
return _("shut down");
case DB_SHUTDOWNED_IN_RECOVERY:
return _("shut down in recovery");
+ case DB_DEMOTING:
+ return _("demoting");
case DB_SHUTDOWNING:
return _("shutting down");
case DB_IN_CRASH_RECOVERY:
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 1cdc3ebaa3..a7805bd219 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -62,6 +62,7 @@ typedef enum
RESTART_COMMAND,
RELOAD_COMMAND,
STATUS_COMMAND,
+ DEMOTE_COMMAND,
PROMOTE_COMMAND,
LOGROTATE_COMMAND,
KILL_COMMAND,
@@ -103,6 +104,7 @@ static char version_file[MAXPGPATH];
static char pid_file[MAXPGPATH];
static char backup_file[MAXPGPATH];
static char promote_file[MAXPGPATH];
+static char demote_file[MAXPGPATH];
static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
@@ -129,6 +131,7 @@ static void do_stop(void);
static void do_restart(void);
static void do_reload(void);
static void do_status(void);
+static void do_demote(void);
static void do_promote(void);
static void do_logrotate(void);
static void do_kill(pgpid_t pid);
@@ -1029,6 +1032,115 @@ do_stop(void)
}
+static void
+do_demote(void)
+{
+ int cnt;
+ FILE *dmtfile;
+ pgpid_t pid;
+ struct stat statbuf;
+
+ pid = get_pgpid(false);
+
+ if (pid == 0) /* no pid file */
+ {
+ write_stderr(_("%s: PID file \"%s\" does not exist\n"), progname, pid_file);
+ write_stderr(_("Is server running?\n"));
+ exit(1);
+ }
+ else if (pid < 0) /* standalone backend, not postmaster */
+ {
+ pid = -pid;
+ write_stderr(_("%s: cannot demote server; "
+ "single-user server is running (PID: %ld)\n"),
+ progname, pid);
+ exit(1);
+ }
+
+ if (shutdown_mode == IMMEDIATE_MODE)
+ {
+ write_stderr(_("%s: cannot demote server using immediate mode"),
+ progname);
+ exit(1);
+ }
+ else if (shutdown_mode == FAST_MODE)
+ snprintf(demote_file, MAXPGPATH, "%s/demote_fast", pg_data);
+ else
+ snprintf(demote_file, MAXPGPATH, "%s/demote", pg_data);
+
+ if ((dmtfile = fopen(demote_file, "w")) == NULL)
+ {
+ write_stderr(_("%s: could not create demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ if (fclose(dmtfile))
+ {
+ write_stderr(_("%s: could not write demote signal file \"%s\": %s\n"),
+ progname, demote_file, strerror(errno));
+ exit(1);
+ }
+
+ sig = SIGUSR1;
+ if (kill((pid_t) pid, sig) != 0)
+ {
+ write_stderr(_("%s: could not send demote signal (PID: %ld): %s\n"), progname, pid,
+ strerror(errno));
+ exit(1);
+ }
+
+ if (!do_wait)
+ {
+ print_msg(_("server demoting\n"));
+ return;
+ }
+ else
+ {
+ /*
+ * FIXME demote
+ * If backup_label exists, an online backup is running. Warn the user
+ * that smart demote will wait for it to finish. However, if the
+ * server is in archive recovery, we're recovering from an online
+ * backup instead of performing one.
+ */
+ if (shutdown_mode == SMART_MODE &&
+ stat(backup_file, &statbuf) == 0 &&
+ get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_("WARNING: online backup mode is active\n"
+ "Demote will not complete until pg_stop_backup() is called.\n\n"));
+ }
+
+ print_msg(_("waiting for server to demote..."));
+
+ for (cnt = 0; cnt < wait_seconds * WAITS_PER_SEC; cnt++)
+ {
+ if (get_control_dbstate() == DB_IN_ARCHIVE_RECOVERY)
+ break;
+
+ if (cnt % WAITS_PER_SEC == 0)
+ print_msg(".");
+ pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
+ }
+
+ if (get_control_dbstate() != DB_IN_ARCHIVE_RECOVERY)
+ {
+ print_msg(_(" failed\n"));
+
+ write_stderr(_("%s: server does not demote\n"), progname);
+ if (shutdown_mode == SMART_MODE)
+ write_stderr(_("HINT: The \"-m fast\" option immediately disconnects sessions rather than\n"
+ "waiting for session-initiated disconnection.\n"));
+ exit(1);
+ }
+ print_msg(_(" done\n"));
+
+ print_msg(_("server demoted\n"));
+ }
+}
+
+
/*
* restart/reload routines
*/
@@ -2447,6 +2559,8 @@ main(int argc, char **argv)
ctl_command = RELOAD_COMMAND;
else if (strcmp(argv[optind], "status") == 0)
ctl_command = STATUS_COMMAND;
+ else if (strcmp(argv[optind], "demote") == 0)
+ ctl_command = DEMOTE_COMMAND;
else if (strcmp(argv[optind], "promote") == 0)
ctl_command = PROMOTE_COMMAND;
else if (strcmp(argv[optind], "logrotate") == 0)
@@ -2554,6 +2668,9 @@ main(int argc, char **argv)
case RELOAD_COMMAND:
do_reload();
break;
+ case DEMOTE_COMMAND:
+ do_demote();
+ break;
case PROMOTE_COMMAND:
do_promote();
break;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 2ca71c3445..4b56f92181 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -53,6 +53,7 @@ extern void RecoverPreparedTransactions(void);
extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
extern void FinishPreparedTransaction(const char *gid, bool isCommit);
+void ShutdownPreparedTransactions(void);
extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
XLogRecPtr end_lsn, RepOriginId origin_id);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8c9cadc6da..b1b1ea67f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -219,18 +219,20 @@ extern bool XLOG_DEBUG;
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
-#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint, but
+#define CHECKPOINT_IS_DEMOTE 0x0002 /* Like shutdown checkpoint, but
+ * issued at end of WAL production */
+#define CHECKPOINT_END_OF_RECOVERY 0x0004 /* Like shutdown checkpoint, but
* issued at end of WAL recovery */
-#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
-#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
-#define CHECKPOINT_FLUSH_ALL 0x0010 /* Flush all pages, including those
+#define CHECKPOINT_IMMEDIATE 0x0008 /* Do it without delays */
+#define CHECKPOINT_FORCE 0x0010 /* Force even if no activity */
+#define CHECKPOINT_FLUSH_ALL 0x0020 /* Flush all pages, including those
* belonging to unlogged tables */
/* These are important to RequestCheckpoint */
-#define CHECKPOINT_WAIT 0x0020 /* Wait for completion */
-#define CHECKPOINT_REQUESTED 0x0040 /* Checkpoint request has been made */
+#define CHECKPOINT_WAIT 0x0040 /* Wait for completion */
+#define CHECKPOINT_REQUESTED 0x0080 /* Checkpoint request has been made */
/* These indicate the cause of a checkpoint request */
-#define CHECKPOINT_CAUSE_XLOG 0x0080 /* XLOG consumption */
-#define CHECKPOINT_CAUSE_TIME 0x0100 /* Elapsed time */
+#define CHECKPOINT_CAUSE_XLOG 0x0100 /* XLOG consumption */
+#define CHECKPOINT_CAUSE_TIME 0x0200 /* Elapsed time */
/*
* Flag bits for the record being inserted, set using XLogSetRecordFlags().
@@ -301,6 +303,7 @@ extern const char *xlog_identify(uint8 info);
extern void issue_xlog_fsync(int fd, XLogSegNo segno);
+extern bool SetLocalRecoveryInProgress(void);
extern bool RecoveryInProgress(void);
extern RecoveryState GetRecoveryState(void);
extern bool HotStandbyActive(void);
@@ -397,4 +400,8 @@ extern SessionBackupState get_backup_status(void);
/* files to signal promotion to primary */
#define PROMOTE_SIGNAL_FILE "promote"
+/* files to signal demotion to standby */
+#define DEMOTE_SIGNAL_FILE "demote"
+#define DEMOTE_FAST_SIGNAL_FILE "demote_fast"
+
#endif /* XLOG_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e..3b30c1d767 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -87,6 +87,7 @@ typedef enum DBState
DB_STARTUP = 0,
DB_SHUTDOWNED,
DB_SHUTDOWNED_IN_RECOVERY,
+ DB_DEMOTING,
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 0a23281ad5..24ba3da013 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -70,7 +70,7 @@ typedef struct
typedef enum CAC_state
{
- CAC_OK, CAC_STARTUP, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
+ CAC_OK, CAC_STARTUP, CAC_DEMOTE, CAC_SHUTDOWN, CAC_RECOVERY, CAC_TOOMANY,
CAC_SUPERUSER
} CAC_state;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..d60804208f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,7 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t DemotePending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1387201382..f1c0a37e76 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -880,6 +880,7 @@ typedef enum
WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
WAIT_EVENT_PROC_SIGNAL_BARRIER,
WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_DEMOTE,
WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT,
WAIT_EVENT_RECOVERY_CONFLICT_TABLESPACE,
WAIT_EVENT_RECOVERY_PAUSE,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..4d4f0ea1dd 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -41,5 +41,6 @@ extern Size CheckpointerShmemSize(void);
extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern void ReqCheckpointDemoteHandler(SIGNAL_ARGS);
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 1c3e9c1999..fa8d64e1a7 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -584,6 +584,8 @@ extern void lock_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len);
extern void lock_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
+extern void lock_twophase_shutdown(TransactionId xid, uint16 info,
+ void *recdata, uint32 len);
extern void lock_twophase_standby_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f3..7264e9a705 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,10 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_DEMOTING, /* ask backends to demote in smart mode */
+ PROCSIG_DEMOTING_FAST, /* ask backends to demote in fast mode */
+ PROCSIG_DEMOTED, /* ask backends to switch to recovery mode */
+ PROCSIG_CHECKPOINTER_DEMOTING, /* ask checkpointer to demote */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index bd30607b07..e5f42f9fec 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -68,6 +68,8 @@ extern void StatementCancelHandler(SIGNAL_ARGS);
extern void FloatExceptionHandler(SIGNAL_ARGS) pg_attribute_noreturn();
extern void RecoveryConflictInterrupt(ProcSignalReason reason); /* called from SIGUSR1
* handler */
+extern void ReqDemoteHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
+extern void ReqDemotedHandler(ProcSignalReason reason); /* called from SIGUSR1 handler */
extern void ProcessClientReadInterrupt(bool blocked);
extern void ProcessClientWriteInterrupt(bool blocked);
diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
index 63fefe5c4c..f761d2c4ef 100644
--- a/src/include/utils/pidfile.h
+++ b/src/include/utils/pidfile.h
@@ -50,6 +50,7 @@
*/
#define PM_STATUS_STARTING "starting" /* still starting up */
#define PM_STATUS_STOPPING "stopping" /* in shutdown sequence */
+#define PM_STATUS_DEMOTING "demoting" /* demote sequence */
#define PM_STATUS_READY "ready " /* ready for connections */
#define PM_STATUS_STANDBY "standby " /* up, won't accept connections */
--
2.20.1
--MP_/6xG3U2Dzv0UR=.BCNK7fiL6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0003-demote-add-pg_demote-function.patch
^ permalink raw reply [nested|flat] 42+ messages in thread
* Move global variables of pgoutput to plugin private scope.
@ 2023-09-19 04:10 Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2023-09-19 04:10 UTC (permalink / raw)
To: PostgreSQL Hackers <[email protected]>
Hi,
Per complain in another thread[1], I started to look into the global
variables in pgoutput.
Currently we have serval global variables in pgoutput, but each of them is
inherently local to an individual pgoutput instance. This could cause issues if
we switch to different output plugin instance in one session and could miss to
reset their value in case of errors. The analysis for each variable is as
follows:
- static HTAB *RelationSyncCache = NULL;
pgoutput creates this hash table under cacheMemoryContext to remember the
relation schemas that have been sent, but it's local to an individual pgoutput
instance, and because it's under global memory context, the hashtable is not
properly cleared in error paths which means it has a risk of being accessed in
a different output plugin instance. This was also mentioned in another thread[2].
So I think we'd better allocate this under output plugin private context.
But note that, instead of completely moving the hash table into the output
plugin private data, we need to to keep the static pointer variable for the map to
be accessed by the syscache callbacks. This is because syscache callbacks won't
be un-registered even after shutting down the output plugin, so we need a
static pointer to cache the map pointer so that callbacks can check it.
- static bool publish_no_origin;
This flag is also local to pgoutput instance, and we didn't reset the flag in
output shutdown callback, so if we consume changes from different slots, then
the second call would reuse the flag value that is set in the first call which
is unexpected. To completely avoid this issue, we think we'd better move this
flag to output plugin private data structure.
Example:
SELECT data FROM pg_logical_slot_peek_binary_changes('isolation_slot_1', NULL, NULL, 'proto_version', '1', 'publication_names', 'pub', 'origin', 'none'); --- Set origin in this call.
SELECT data FROM pg_logical_slot_peek_binary_changes('isolation_slot_2', NULL, NULL, 'proto_version', '1', 'publication_names', 'pub'); -- Didn't set origin, but will reuse the origin flag in the first call.
- static bool in_streaming;
While on it, I feel we can also move this flag to private data, although I didn't
see problems for this one.
- static bool publications_valid;
I thought we need to move this to private data as well, but we need to access this in a
syscache callback, which means we need to keep the static variable.
Attach the patches to change in_streaming, publish_no_origin and RelationSyncCache.
Suggestions and comments are welcome.
[1] https://www.postgresql.org/message-id/20230821182732.t3qc75i5s5xvovls%40awork3.anarazel.de
[2] https://www.postgresql.org/message-id/CAA4eK1LJ%3DCSsxETs5ydqP58OiWPiwodx%3DJqw89LQ7fMrRWqK9w%40mail...
Best Regards,
Hou Zhijie
Attachments:
[application/octet-stream] v1-0003-Move-in_streaming-to-output-private-data.patch (6.0K, ../../OS0PR01MB571690EF24F51F51EFFCBB0E94FAA@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v1-0003-Move-in_streaming-to-output-private-data.patch)
download | inline diff:
From c4ce199d6f8803ab4695aa44d25142235b2f6aac Mon Sep 17 00:00:00 2001
From: Hou Zhijie <[email protected]>
Date: Wed, 13 Sep 2023 20:22:02 +0800
Subject: [PATCH v1 3/3] Move in_streaming to output private data
---
src/backend/replication/pgoutput/pgoutput.c | 34 +++++++++++----------
src/include/replication/pgoutput.h | 2 ++
2 files changed, 20 insertions(+), 16 deletions(-)
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 89789c298f..9f076d99c3 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -81,7 +81,6 @@ static void pgoutput_stream_prepare_txn(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr prepare_lsn);
static bool publications_valid;
-static bool in_streaming;
static List *LoadPublications(List *pubnames);
static void publication_invalidation_cb(Datum arg, int cacheid,
@@ -479,9 +478,6 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("streaming requested, but not supported by output plugin")));
- /* Also remember we're currently not streaming any transaction. */
- in_streaming = false;
-
/*
* Here, we just check whether the two-phase option is passed by
* plugin and decide whether to enable it at later point of time. It
@@ -679,6 +675,7 @@ maybe_send_schema(LogicalDecodingContext *ctx,
ReorderBufferChange *change,
Relation relation, RelationSyncEntry *relentry)
{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
bool schema_sent;
TransactionId xid = InvalidTransactionId;
TransactionId topxid = InvalidTransactionId;
@@ -691,7 +688,7 @@ maybe_send_schema(LogicalDecodingContext *ctx,
* If we're not in a streaming block, just use InvalidTransactionId and
* the write methods will not include it.
*/
- if (in_streaming)
+ if (data->in_streaming)
xid = change->txn->xid;
if (rbtxn_is_subtxn(change->txn))
@@ -711,7 +708,7 @@ maybe_send_schema(LogicalDecodingContext *ctx,
* doing that we need to study its impact on the case where we have a mix
* of streaming and non-streaming transactions.
*/
- if (in_streaming)
+ if (data->in_streaming)
schema_sent = get_schema_sent_in_streamed_txn(relentry, topxid);
else
schema_sent = relentry->schema_sent;
@@ -735,7 +732,7 @@ maybe_send_schema(LogicalDecodingContext *ctx,
send_relation_and_attrs(relation, xid, ctx, relentry->columns);
- if (in_streaming)
+ if (data->in_streaming)
set_schema_sent_in_streamed_txn((PGOutputData *) ctx->output_plugin_private,
relentry, topxid);
else
@@ -1422,7 +1419,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
* their association and on aborts, it can discard the corresponding
* changes.
*/
- if (in_streaming)
+ if (data->in_streaming)
xid = change->txn->xid;
relentry = get_rel_sync_entry(data, relation);
@@ -1571,7 +1568,7 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
TransactionId xid = InvalidTransactionId;
/* Remember the xid for the change in streaming mode. See pgoutput_change. */
- if (in_streaming)
+ if (data->in_streaming)
xid = change->txn->xid;
old = MemoryContextSwitchTo(data->context);
@@ -1640,7 +1637,7 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
* Remember the xid for the message in streaming mode. See
* pgoutput_change.
*/
- if (in_streaming)
+ if (data->in_streaming)
xid = txn->xid;
/*
@@ -1743,10 +1740,11 @@ static void
pgoutput_stream_start(struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
bool send_replication_origin = txn->origin_id != InvalidRepOriginId;
/* we can't nest streaming of transactions */
- Assert(!in_streaming);
+ Assert(!data->in_streaming);
/*
* If we already sent the first stream for this transaction then don't
@@ -1764,7 +1762,7 @@ pgoutput_stream_start(struct LogicalDecodingContext *ctx,
OutputPluginWrite(ctx, true);
/* we're streaming a chunk of transaction now */
- in_streaming = true;
+ data->in_streaming = true;
}
/*
@@ -1774,15 +1772,17 @@ static void
pgoutput_stream_stop(struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+
/* we should be streaming a transaction */
- Assert(in_streaming);
+ Assert(data->in_streaming);
OutputPluginPrepareWrite(ctx, true);
logicalrep_write_stream_stop(ctx->out);
OutputPluginWrite(ctx, true);
/* we've stopped streaming a transaction */
- in_streaming = false;
+ data->in_streaming = false;
}
/*
@@ -1802,7 +1802,7 @@ pgoutput_stream_abort(struct LogicalDecodingContext *ctx,
* The abort should happen outside streaming block, even for streamed
* transactions. The transaction has to be marked as streamed, though.
*/
- Assert(!in_streaming);
+ Assert(!data->in_streaming);
/* determine the toplevel transaction */
toptxn = rbtxn_get_toptxn(txn);
@@ -1827,11 +1827,13 @@ pgoutput_stream_commit(struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr commit_lsn)
{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+
/*
* The commit should happen outside streaming block, even for streamed
* transactions. The transaction has to be marked as streamed, though.
*/
- Assert(!in_streaming);
+ Assert(!data->in_streaming);
Assert(rbtxn_is_streamed(txn));
OutputPluginUpdateProgress(ctx, false);
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index 6f39e72252..d928010248 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -21,6 +21,8 @@ typedef struct PGOutputData
* allocations */
MemoryContext cachectx; /* private memory context for cache data */
+ bool in_streaming;
+
/* client-supplied info: */
uint32 protocol_version;
List *publication_names;
--
2.30.0.windows.2
[application/octet-stream] v1-0001-Allocate-RelationSyncCache-of-pgoutput-under-priv.patch (5.2K, ../../OS0PR01MB571690EF24F51F51EFFCBB0E94FAA@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v1-0001-Allocate-RelationSyncCache-of-pgoutput-under-priv.patch)
download | inline diff:
From fb423bb016482e8bf553f98485346540602f7bae Mon Sep 17 00:00:00 2001
From: Hou Zhijie <[email protected]>
Date: Fri, 25 Aug 2023 17:07:13 +0800
Subject: [PATCH v1 1/3] Allocate RelationSyncCache of pgoutput under private
memory context
The pgoutput module creates a map under cacheMemoryContext to remember the
relation schemas that have been sent.
However, this map is inherently local to an individual pgoutput instance.
Additionally, while the map is adequately cleaned up when shutting down the
output plugin, it is not properly cleared in error paths. Consequently,
subsequent retries may access the outdated map.
To improve it, the patch allocates the map under the memory context specific to
the output plugin and registers a memory context reset callback to clean it.
Note: Instead of completely moving the map variable into the output plugin
private data, we have to keep the static pointer variable for the map to be
accessed by the syscache callbacks. This is because syscache callbacks won't be
un-registered even after shutting down the output plugin, so we need a static
pointer to cache the map pointer so that callbacks can check it.
---
src/backend/replication/pgoutput/pgoutput.c | 43 +++++++++++++++------
1 file changed, 31 insertions(+), 12 deletions(-)
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b08ca55041..b3d515b59f 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -219,7 +219,8 @@ static RelationSyncEntry *get_rel_sync_entry(PGOutputData *data,
static void rel_sync_cache_relation_cb(Datum arg, Oid relid);
static void rel_sync_cache_publication_cb(Datum arg, int cacheid,
uint32 hashvalue);
-static void set_schema_sent_in_streamed_txn(RelationSyncEntry *entry,
+static void set_schema_sent_in_streamed_txn(PGOutputData *data,
+ RelationSyncEntry *entry,
TransactionId xid);
static bool get_schema_sent_in_streamed_txn(RelationSyncEntry *entry,
TransactionId xid);
@@ -520,7 +521,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
}
/* Initialize relation schema cache. */
- init_rel_sync_cache(CacheMemoryContext);
+ init_rel_sync_cache(data->cachectx);
}
else
{
@@ -736,7 +737,8 @@ maybe_send_schema(LogicalDecodingContext *ctx,
send_relation_and_attrs(relation, xid, ctx, relentry->columns);
if (in_streaming)
- set_schema_sent_in_streamed_txn(relentry, topxid);
+ set_schema_sent_in_streamed_txn((PGOutputData *) ctx->output_plugin_private,
+ relentry, topxid);
else
relentry->schema_sent = true;
}
@@ -1157,7 +1159,7 @@ init_tuple_slot(PGOutputData *data, Relation relation,
TupleDesc outdesc = RelationGetDescr(ancestor);
/* Map must live as long as the session does. */
- oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ oldctx = MemoryContextSwitchTo(data->cachectx);
entry->attrmap = build_attrmap_by_name_if_req(indesc, outdesc, false);
@@ -1689,11 +1691,11 @@ pgoutput_origin_filter(LogicalDecodingContext *ctx,
static void
pgoutput_shutdown(LogicalDecodingContext *ctx)
{
- if (RelationSyncCache)
- {
- hash_destroy(RelationSyncCache);
- RelationSyncCache = NULL;
- }
+ /*
+ * Don't need to destroy the hash table as it will be cleaned along with
+ * logical decoding memory context.
+ */
+ RelationSyncCache = NULL;
}
/*
@@ -1858,6 +1860,15 @@ pgoutput_stream_prepare_txn(LogicalDecodingContext *ctx,
OutputPluginWrite(ctx, true);
}
+/*
+ * Memory context reset callback for clearing the RelationSyncCache.
+ */
+static void
+reset_rel_sync_cache(void *arg)
+{
+ RelationSyncCache = NULL;
+}
+
/*
* Initialize the relation schema sync cache for a decoding session.
*
@@ -1869,12 +1880,19 @@ static void
init_rel_sync_cache(MemoryContext cachectx)
{
HASHCTL ctl;
+ MemoryContextCallback *mcallback;
static bool relation_callbacks_registered = false;
/* Nothing to do if hash table already exists */
if (RelationSyncCache != NULL)
return;
+ /* Ensure that the hash table is reset in error paths. */
+ mcallback = MemoryContextAllocZero(cachectx,
+ sizeof(MemoryContextCallback));
+ mcallback->func = reset_rel_sync_cache;
+ MemoryContextRegisterResetCallback(cachectx, mcallback);
+
/* Make a new hash table for the cache */
ctl.keysize = sizeof(Oid);
ctl.entrysize = sizeof(RelationSyncEntry);
@@ -1930,11 +1948,12 @@ get_schema_sent_in_streamed_txn(RelationSyncEntry *entry, TransactionId xid)
* of the relation.
*/
static void
-set_schema_sent_in_streamed_txn(RelationSyncEntry *entry, TransactionId xid)
+set_schema_sent_in_streamed_txn(PGOutputData *data, RelationSyncEntry *entry,
+ TransactionId xid)
{
MemoryContext oldctx;
- oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ oldctx = MemoryContextSwitchTo(data->cachectx);
entry->streamed_txns = lappend_xid(entry->streamed_txns, xid);
@@ -2005,7 +2024,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
/* Reload publications if needed before use. */
if (!publications_valid)
{
- oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ oldctx = MemoryContextSwitchTo(data->cachectx);
if (data->publications)
{
list_free_deep(data->publications);
--
2.30.0.windows.2
[application/octet-stream] v1-0002-Maintain-publish_no_origin-in-output-plugin-priva.patch (2.7K, ../../OS0PR01MB571690EF24F51F51EFFCBB0E94FAA@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v1-0002-Maintain-publish_no_origin-in-output-plugin-priva.patch)
download | inline diff:
From 507d225e148e3e1ee642b7bf4e8b326f52aa474c Mon Sep 17 00:00:00 2001
From: Hou Zhijie <[email protected]>
Date: Fri, 25 Aug 2023 17:29:35 +0800
Subject: [PATCH v1 2/3] Maintain publish_no_origin in output plugin private
data
The pgoutput module uses a global variable(publish_no_origin) to cache the
action for the origin filter. But we only initialize publish_no_origin when
user specifies the "origin" in the output paramters which means we could refer
to an uninitialized variable if user didn't specify the paramter.
Besides, we don't reset the flag when shutting down the output plugin, so
subsequent retries may access the previous publish_no_origin value.
To improve it, the patch stores the map within the private data of the output
plugin so that it will get initialized and reset along with the output plugin
context.
---
src/backend/replication/pgoutput/pgoutput.c | 9 +++++----
src/include/replication/pgoutput.h | 1 +
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b3d515b59f..89789c298f 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -82,7 +82,6 @@ static void pgoutput_stream_prepare_txn(LogicalDecodingContext *ctx,
static bool publications_valid;
static bool in_streaming;
-static bool publish_no_origin;
static List *LoadPublications(List *pubnames);
static void publication_invalidation_cb(Datum arg, int cacheid,
@@ -390,9 +389,9 @@ parse_output_parameters(List *options, PGOutputData *data)
data->origin = defGetString(defel);
if (pg_strcasecmp(data->origin, LOGICALREP_ORIGIN_NONE) == 0)
- publish_no_origin = true;
+ data->publish_no_origin = true;
else if (pg_strcasecmp(data->origin, LOGICALREP_ORIGIN_ANY) == 0)
- publish_no_origin = false;
+ data->publish_no_origin = false;
else
ereport(ERROR,
errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -1675,7 +1674,9 @@ static bool
pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id)
{
- if (publish_no_origin && origin_id != InvalidRepOriginId)
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+
+ if (data->publish_no_origin && origin_id != InvalidRepOriginId)
return true;
return false;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index b4a8015403..6f39e72252 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -30,6 +30,7 @@ typedef struct PGOutputData
bool messages;
bool two_phase;
char *origin;
+ bool publish_no_origin;
} PGOutputData;
#endif /* PGOUTPUT_H */
--
2.30.0.windows.2
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Move global variables of pgoutput to plugin private scope.
@ 2023-09-19 05:43 Michael Paquier <[email protected]>
parent: Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Michael Paquier @ 2023-09-19 05:43 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Tue, Sep 19, 2023 at 04:10:39AM +0000, Zhijie Hou (Fujitsu) wrote:
> Currently we have serval global variables in pgoutput, but each of them is
> inherently local to an individual pgoutput instance. This could cause issues if
> we switch to different output plugin instance in one session and could miss to
> reset their value in case of errors. The analysis for each variable is as
> follows:
(Moved the last block of the message as per the relationship between
RelationSyncCache and publications_valid).
> - static HTAB *RelationSyncCache = NULL;
>
> pgoutput creates this hash table under cacheMemoryContext to remember the
> relation schemas that have been sent, but it's local to an individual pgoutput
> instance, and because it's under global memory context, the hashtable is not
> properly cleared in error paths which means it has a risk of being accessed in
> a different output plugin instance. This was also mentioned in another thread[2].
>
> So I think we'd better allocate this under output plugin private context.
>
> But note that, instead of completely moving the hash table into the output
> plugin private data, we need to to keep the static pointer variable for the map to
> be accessed by the syscache callbacks. This is because syscache callbacks won't
> be un-registered even after shutting down the output plugin, so we need a
> static pointer to cache the map pointer so that callbacks can check it.
>
> - static bool publications_valid;
>
> I thought we need to move this to private data as well, but we need to access this in a
> syscache callback, which means we need to keep the static variable.
FWIW, I think that keeping publications_valid makes the code kind of
confusing once 0001 is applied, because this makes the handling of the
cached data for relations and publications even more inconsistent than
it is now, with a mixed bag of two different logics caused by the
relationship between the synced relation cache and the publication
cache: RelationSyncCache tracks if relations should be rebuilt, while
publications_valid does it for the publication data, but both are
still static and could be shared by multiple pgoutput contexts. On
top of that, publications_valid is hidden at the top of pgoutput.c
within a bunch of declarations and no comments to explain why it's
here (spoiler: to handle the cache rebuilds with its reset in the
cache callback).
I agree that CacheMemoryContext is not really a good idea to cache the
data only proper to a pgoutput session and that tracking a context in
the output data makes the whole cleanup attractive, but it also seems
to me that we should avoid entirely the use of relcache callbacks if
the intention is to have one RelationSyncEntry per pgoutput. The
patch does something different than HEAD and than having one
RelationSyncEntry per pgoutout: RelationSyncEntry can reference
*everything*, with its data stored in multiple memory contexts as of
one per pgoutput. It looks like RelationSyncEntry should be a list
or a hash table, at least, so as it can refer to multiple pgoutput
states. Knowing that a session can only use one replication slot with
MyReplicationSlot, not sure that's worth bothering with. As a whole,
0001 with its changes for RelationSyncCache don't seem like an
improvement to me.
> - static bool publish_no_origin;
>
> This flag is also local to pgoutput instance, and we didn't reset the flag in
> output shutdown callback, so if we consume changes from different slots, then
> the second call would reuse the flag value that is set in the first call which
> is unexpected. To completely avoid this issue, we think we'd better move this
> flag to output plugin private data structure.
Yep, that's incorrect.
> - static bool in_streaming;
>
> While on it, I feel we can also move this flag to private data, although I didn't
> see problems for this one.
Moving this one to the private state data makes sense to me, as it
tracks the streaming of one PGOutputData.
Note that we name twice RelSchemaSyncCache in the code, but it does
not exist..
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 42+ messages in thread
* RE: Move global variables of pgoutput to plugin private scope.
@ 2023-09-26 13:49 Zhijie Hou (Fujitsu) <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2023-09-26 13:49 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Tuesday, September 19, 2023 1:44 PM Michael Paquier <[email protected]> wrote:
>
> On Tue, Sep 19, 2023 at 04:10:39AM +0000, Zhijie Hou (Fujitsu) wrote:
> > Currently we have serval global variables in pgoutput, but each of
> > them is inherently local to an individual pgoutput instance. This
> > could cause issues if we switch to different output plugin instance in
> > one session and could miss to reset their value in case of errors. The
> > analysis for each variable is as
> > follows:
>
> (Moved the last block of the message as per the relationship between
> RelationSyncCache and publications_valid).
>
> > - static HTAB *RelationSyncCache = NULL;
> >
> > pgoutput creates this hash table under cacheMemoryContext to remember
> > the relation schemas that have been sent, but it's local to an
> > individual pgoutput instance, and because it's under global memory
> > context, the hashtable is not properly cleared in error paths which
> > means it has a risk of being accessed in a different output plugin instance.
> This was also mentioned in another thread[2].
> >
> > So I think we'd better allocate this under output plugin private context.
> >
> > But note that, instead of completely moving the hash table into the
> > output plugin private data, we need to to keep the static pointer
> > variable for the map to be accessed by the syscache callbacks. This is
> > because syscache callbacks won't be un-registered even after shutting
> > down the output plugin, so we need a static pointer to cache the map pointer
> so that callbacks can check it.
> >
> > - static bool publications_valid;
> >
> > I thought we need to move this to private data as well, but we need to
> > access this in a syscache callback, which means we need to keep the static
> variable.
>
> FWIW, I think that keeping publications_valid makes the code kind of confusing
> once 0001 is applied, because this makes the handling of the cached data for
> relations and publications even more inconsistent than it is now, with a mixed
> bag of two different logics caused by the relationship between the synced
> relation cache and the publication
> cache: RelationSyncCache tracks if relations should be rebuilt, while
> publications_valid does it for the publication data, but both are still static and
> could be shared by multiple pgoutput contexts. On top of that,
> publications_valid is hidden at the top of pgoutput.c within a bunch of
> declarations and no comments to explain why it's here (spoiler: to handle the
> cache rebuilds with its reset in the cache callback).
>
> I agree that CacheMemoryContext is not really a good idea to cache the data
> only proper to a pgoutput session and that tracking a context in the output
> data makes the whole cleanup attractive, but it also seems to me that we
> should avoid entirely the use of relcache callbacks if the intention is to have one
> RelationSyncEntry per pgoutput. The patch does something different than
> HEAD and than having one RelationSyncEntry per pgoutout: RelationSyncEntry
> can reference *everything*, with its data stored in multiple memory contexts as
> of one per pgoutput. It looks like RelationSyncEntry should be a list or a hash
> table, at least, so as it can refer to multiple pgoutput states. Knowing that a
> session can only use one replication slot with MyReplicationSlot, not sure that's
> worth bothering with. As a whole,
> 0001 with its changes for RelationSyncCache don't seem like an improvement
> to me.
>
Thanks for your comments. Currently, I am not sure how to avoid the use of the
syscache callback functions, So I think the change for RelationSyncCache needs
more thought and I will retry later if I find another way.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 42+ messages in thread
end of thread, other threads:[~2023-09-26 13:49 UTC | newest]
Thread overview: 42+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2020-04-10 16:01 [PATCH 2/4] demote: support demoting instance from production to standby Jehan-Guillaume de Rorthais <[email protected]>
2023-09-19 04:10 Move global variables of pgoutput to plugin private scope. Zhijie Hou (Fujitsu) <[email protected]>
2023-09-19 05:43 ` Re: Move global variables of pgoutput to plugin private scope. Michael Paquier <[email protected]>
2023-09-26 13:49 ` RE: Move global variables of pgoutput to plugin private scope. Zhijie Hou (Fujitsu) <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox