pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Bossart, Nathan <bossartn@amazon.com>
To: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: x4mmm@yandex-team.ru <x4mmm@yandex-team.ru>
Cc: a.lubennikova@postgrespro.ru <a.lubennikova@postgrespro.ru>
Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>
Cc: matsumura.ryo@fujitsu.com <matsumura.ryo@fujitsu.com>
Cc: masao.fujii@gmail.com <masao.fujii@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: archive status ".ready" files may be created too early
Date: Tue, 17 Aug 2021 21:09:52 +0000
Message-ID: <2A53AFB0-9A44-4EFE-BA83-072D000D9319@amazon.com> (raw)
In-Reply-To: <202108172023.b52p6convvvd@alvherre.pgsql>
References: <867B852E-E892-4DD7-8E29-CE74FDBBA234@amazon.com>
	<202108172023.b52p6convvvd@alvherre.pgsql>

On 8/17/21, 1:24 PM, "alvherre@alvh.no-ip.org" <alvherre@alvh.no-ip.org> wrote:
> The thing I still don't understand about this patch is why we call
> RegisterSegmentBoundaryEntry and NotifySegmentsReadyForArchive in
> XLogInsertRecord.
>
> My model concept of this would have these routines called only in
> XLogFlush / XLogWrite, which are conceptually about transferring data
> from in-memory WAL buffers into the on-disk WAL store (pg_xlog files).
> As I understand, XLogInsertRecord is about copying bytes from the
> high-level operation (heap insert etc) into WAL buffers.  So doing the
> register/notify dance in both places should be redundant and
> unnecessary.

The main reason for registering the boundaries in XLogInsertRecord()
is that it has the required information about the WAL record
boundaries.  I do not think that XLogWrite() has this information.
If we assumed that write requests always pointed to record boundaries,
we could probably just move the XLogArchiveNotifySeg() calls to the
end of XLogWrite(), which is what my original patch [0] did.

> In the NotifySegmentsReadyForArchive() call at the bottom of XLogWrite,
> we use flushed=InvalidXLogRecPtr.  Why is that?  Surely we can use
> LogwrtResult.Flush, just like in the other callsite there?  (If we're
> covering for somebody advancing FlushRecPtr concurrently, I think we
> add a comment to explain that.  But why do we need to do that in the
> first place?)

Good point.  I did this in the new version of the patch.

Nathan

[0] https://postgr.es/m/CBDDFA01-6E40-46BB-9F98-9340F4379505%40amazon.com



Attachments:

v8-0001-Avoid-creating-archive-status-.ready-files-too-ea.patch (application/octet-stream, 19.0K) download From c7a81ebc018dd1e5cb8fb899d4b22932997cb76f Mon Sep 17 00:00:00 2001 From: Nathan Bossart <bossartn@amazon.com> Date: Tue, 17 Aug 2021 20:52:51 +0000 Subject: [PATCH v8 1/1] Avoid creating archive status ".ready" files too early. WAL records may span multiple segments, but XLogWrite() does not wait for the entire record to be written out to disk before creating archive status files. Instead, as soon as the last WAL page of the segment is written, the archive status file will be created. If PostgreSQL crashes before it is able to write the rest of the record, it will end up reusing segments that have already been marked as ready-for-archival. However, the archiver process may have already processed the old version of the segment, so the wrong version of the segment may be backed-up. This backed-up segment will cause operations such as point-in-time restores to fail. To fix this, we keep track of records that span across segments and ensure that segments are only marked ready-for-archival once such records have been completely written to disk. --- src/backend/access/transam/timeline.c | 2 +- src/backend/access/transam/xlog.c | 286 ++++++++++++++++++++++++++++++- src/backend/access/transam/xlogarchive.c | 14 +- src/backend/replication/walreceiver.c | 6 +- src/backend/storage/lmgr/lwlocknames.txt | 1 + src/include/access/xlogarchive.h | 4 +- src/include/access/xlogdefs.h | 1 + 7 files changed, 292 insertions(+), 22 deletions(-) diff --git a/src/backend/access/transam/timeline.c b/src/backend/access/transam/timeline.c index 8d0903c175..acd5c2431d 100644 --- a/src/backend/access/transam/timeline.c +++ b/src/backend/access/transam/timeline.c @@ -452,7 +452,7 @@ writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI, if (XLogArchivingActive()) { TLHistoryFileName(histfname, newTLI); - XLogArchiveNotify(histfname); + XLogArchiveNotify(histfname, true); } } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index e51a7a749d..eaf2dd8ab1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -512,6 +512,13 @@ typedef enum ExclusiveBackupState */ static SessionBackupState sessionBackupState = SESSION_BACKUP_NONE; +/* entries for SegmentBoundaryMap, used to mark segments ready for archival */ +typedef struct SegmentBoundaryEntry +{ + XLogSegNo seg; /* must be first */ + XLogRecPtr pos; +} SegmentBoundaryEntry; + /* * Shared state data for WAL insertion. */ @@ -723,6 +730,12 @@ typedef struct XLogCtlData */ XLogRecPtr lastFpwDisableRecPtr; + /* + * The last segment we've marked ready for archival. Protected by + * info_lck. + */ + XLogSegNo lastNotifiedSeg; + slock_t info_lck; /* locks shared variables shown above */ } XLogCtlData; @@ -736,6 +749,12 @@ static WALInsertLockPadded *WALInsertLocks = NULL; */ static ControlFileData *ControlFile = NULL; +/* + * Segment boundary map, used for marking segments as ready for archival. + * Protected by SegmentBoundaryLock. + */ +static HTAB *SegmentBoundaryMap = NULL; + /* * Calculate the amount of space left on the page after 'endptr'. Beware * multiple evaluation! @@ -962,6 +981,14 @@ static XLogRecPtr XLogBytePosToRecPtr(uint64 bytepos); static XLogRecPtr XLogBytePosToEndRecPtr(uint64 bytepos); static uint64 XLogRecPtrToBytePos(XLogRecPtr ptr); static void checkXLogConsistency(XLogReaderState *record); +static void RegisterSegmentBoundaryEntry(XLogSegNo seg, XLogRecPtr pos); +static void NotifySegmentsReadyForArchive(XLogRecPtr flushRecPtr); +static XLogSegNo GetLastNotifiedSegment(void); +static void SetLastNotifiedSegment(XLogSegNo seg); +static bool GetLatestSegmentBoundary(XLogSegNo last_notified, + XLogRecPtr flushed, + XLogSegNo *latest_boundary_seg); +static void RemoveSegmentBoundariesUpTo(XLogSegNo seg); static void WALInsertLockAcquire(void); static void WALInsertLockAcquireExclusive(void); @@ -1158,6 +1185,9 @@ XLogInsertRecord(XLogRecData *rdata, */ if (StartPos / XLOG_BLCKSZ != EndPos / XLOG_BLCKSZ) { + XLogSegNo StartSeg; + XLogSegNo EndSeg; + SpinLockAcquire(&XLogCtl->info_lck); /* advance global request to include new block(s) */ if (XLogCtl->LogwrtRqst.Write < EndPos) @@ -1165,12 +1195,36 @@ XLogInsertRecord(XLogRecData *rdata, /* update local result copy while I have the chance */ LogwrtResult = XLogCtl->LogwrtResult; SpinLockRelease(&XLogCtl->info_lck); + + /* + * If we crossed the segment boundary, record it. This is used to + * ensure that segments are not marked ready for archival before the + * entire record has been flushed to disk. + * + * Note that we do not use XLByteToPrevSeg() for determining the ending + * segment. This is done so that a record that fits perfectly into the + * end of the segment is marked ready for archival as soon as the + * flushed pointer jumps to the next segment. + */ + XLByteToSeg(StartPos, StartSeg, wal_segment_size); + XLByteToSeg(EndPos, EndSeg, wal_segment_size); + + if (StartSeg != EndSeg && XLogArchivingActive()) + { + RegisterSegmentBoundaryEntry(EndSeg, EndPos); + + /* + * There's a chance that the record was already flushed to disk and + * we missed marking segments as ready for archive, so try to do + * that now. + */ + NotifySegmentsReadyForArchive(GetFlushRecPtr()); + } } /* * If this was an XLOG_SWITCH record, flush the record and the empty - * padding space that fills the rest of the segment, and perform - * end-of-segment actions (eg, notifying archiver). + * padding space that fills the rest of the segment. */ if (isLogSwitch) { @@ -1264,6 +1318,192 @@ XLogInsertRecord(XLogRecData *rdata, return EndPos; } +/* + * RegisterSegmentBoundaryEntry + * + * This enters a new entry into the segment boundary map, which is used for + * determing when it is safe to mark a segment as ready for archival. An entry + * with the given key (the segment number) must not already exist in the map. + * Also, the caller is responsible for ensuring that XLByteToSeg() would return + * the same segment number for the given record pointer. + */ +static void +RegisterSegmentBoundaryEntry(XLogSegNo seg, XLogRecPtr pos) +{ + SegmentBoundaryEntry *entry; + bool found; + + LWLockAcquire(SegmentBoundaryLock, LW_EXCLUSIVE); + + entry = (SegmentBoundaryEntry *) hash_search(SegmentBoundaryMap, + (void *) &seg, HASH_ENTER, + &found); + if (found) + elog(ERROR, "entry for segment already exists"); + + entry->pos = pos; + + LWLockRelease(SegmentBoundaryLock); +} + +/* + * NotifySegmentsReadyForArchive + * + * This function marks segments as ready for archival, given that it is safe to + * do so. It is safe to call this function repeatedly, even if nothing has + * changed since the last time it was called. + */ +static void +NotifySegmentsReadyForArchive(XLogRecPtr flushRecPtr) +{ + XLogSegNo flushed_seg; + XLogSegNo latest_boundary_seg; + XLogSegNo last_notified; + + /* + * We first do a quick sanity check to see if we can bail out without taking + * the SegmentBoundaryLock at all. It is expected that this function will + * run frequently and that it will need to do nothing the vast majority of + * the time. + * + * Specifically, we bail out if we've already marked the segment prior to + * the segment that contains flushRecPtr as ready for archival. We + * intentionally use XLByteToSeg() instead of XLByteToPrevSeg() so that we + * don't skip notifying when a record fits perfectly into the end of a + * segment. (flushRecPtr should point to the first byte of the record + * _after_ the one that is known to be flushed to disk.) + */ + last_notified = GetLastNotifiedSegment(); + XLByteToSeg(flushRecPtr, flushed_seg, wal_segment_size); + if (last_notified >= flushed_seg - 1) + return; + + LWLockAcquire(SegmentBoundaryLock, LW_EXCLUSIVE); + + /* Reobtain lastNotifiedSeg in case someone else changed it. */ + last_notified = GetLastNotifiedSegment(); + + /* Retrieve the latest segment boundary to use for notifying segments. */ + if (GetLatestSegmentBoundary(last_notified, flushRecPtr, &latest_boundary_seg)) + { + /* + * Update shared memory and discard segment boundaries that are no + * longer needed. + * + * It is safe to update shared memory before we attempt to create the + * .ready files. If our calls to XLogArchiveNotifySeg() fail, + * RemoveOldXlogFiles() will retry it as needed. + */ + SetLastNotifiedSegment(latest_boundary_seg - 1); + RemoveSegmentBoundariesUpTo(latest_boundary_seg); + + LWLockRelease(SegmentBoundaryLock); + + /* + * Notify archiver about segments that are ready for archival (by + * creating the corresponding .ready files). + */ + for (XLogSegNo seg = last_notified + 1; seg < latest_boundary_seg; seg++) + XLogArchiveNotifySeg(seg, false); + + PgArchWakeup(); + } + else + LWLockRelease(SegmentBoundaryLock); +} + +/* + * GetLatestSegmentBoundary + * + * This function finds the latest segment boundary in SegmentBoundaryMap that is + * less than or equal to the given "flushed" pointer and beyond the last + * notified segment. If such a segment is found, latest_boundary_seg is + * populated and true is returned. Otherwise, false is returned. + */ +static bool +GetLatestSegmentBoundary(XLogSegNo last_notified, XLogRecPtr flushed, + XLogSegNo *latest_boundary_seg) +{ + XLogSegNo flushed_seg; + XLogSegNo seg; + + Assert(LWLockHeldByMe(SegmentBoundaryLock)); + Assert(latest_boundary_seg != NULL); + + XLByteToSeg(flushed, flushed_seg, wal_segment_size); + + for (seg = flushed_seg; seg > last_notified; seg--) + { + SegmentBoundaryEntry *entry; + + entry = (SegmentBoundaryEntry *) hash_search(SegmentBoundaryMap, + (void *) &seg, HASH_FIND, + NULL); + + if (entry != NULL && flushed >= entry->pos) + { + *latest_boundary_seg = entry->seg; + return true; + } + } + + return false; +} + +/* + * RemoveSegmentBoundariesUpTo + * + * This function removes all entries in the SegmentBoundaryMap with segment + * numbers up to and including seg. + */ +static void +RemoveSegmentBoundariesUpTo(XLogSegNo seg) +{ + SegmentBoundaryEntry *entry; + HASH_SEQ_STATUS status; + + Assert(LWLockHeldByMeInMode(SegmentBoundaryLock, LW_EXCLUSIVE)); + + hash_seq_init(&status, SegmentBoundaryMap); + + while ((entry = (SegmentBoundaryEntry *) hash_seq_search(&status)) != NULL) + { + if (entry->seg <= seg) + (void) hash_search(SegmentBoundaryMap, (void *) &entry->seg, + HASH_REMOVE, NULL); + } +} + +/* + * GetLastNotifiedSegment + * + * Retrieves last notified segment from shared memory. + */ +XLogSegNo +GetLastNotifiedSegment(void) +{ + XLogSegNo seg; + + SpinLockAcquire(&XLogCtl->info_lck); + seg = XLogCtl->lastNotifiedSeg; + SpinLockRelease(&XLogCtl->info_lck); + + return seg; +} + +/* + * SetLastNotifiedSegment + * + * Sets last notified segment in shared memory. + */ +static void +SetLastNotifiedSegment(XLogSegNo seg) +{ + SpinLockAcquire(&XLogCtl->info_lck); + XLogCtl->lastNotifiedSeg = seg; + SpinLockRelease(&XLogCtl->info_lck); +} + /* * Reserves the right amount of space for a record of given size from the WAL. * *StartPos is set to the beginning of the reserved section, *EndPos to @@ -2586,11 +2826,13 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible) * later. Doing it here ensures that one and only one backend will * perform this fsync. * - * This is also the right place to notify the Archiver that the - * segment is ready to copy to archival storage, and to update the - * timer for archive_timeout, and to signal for a checkpoint if - * too many logfile segments have been used since the last - * checkpoint. + * If WAL archiving is active, we attempt to notify the archiver of + * any segments that are now ready for archival. + * + * This is also the right place to update the timer for + * archive_timeout and to signal for a checkpoint if too many + * logfile segments have been used since the last checkpoint. + * */ if (finishing_seg) { @@ -2602,7 +2844,7 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible) LogwrtResult.Flush = LogwrtResult.Write; /* end of page */ if (XLogArchivingActive()) - XLogArchiveNotifySeg(openLogSegNo); + NotifySegmentsReadyForArchive(LogwrtResult.Flush); XLogCtl->lastSegSwitchTime = (pg_time_t) time(NULL); XLogCtl->lastSegSwitchLSN = LogwrtResult.Flush; @@ -2690,6 +2932,9 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible) XLogCtl->LogwrtRqst.Flush = LogwrtResult.Flush; SpinLockRelease(&XLogCtl->info_lck); } + + if (XLogArchivingActive()) + NotifySegmentsReadyForArchive(LogwrtResult.Flush); } /* @@ -5117,6 +5362,9 @@ XLOGShmemSize(void) /* and the buffers themselves */ size = add_size(size, mul_size(XLOG_BLCKSZ, XLOGbuffers)); + /* stuff for marking segments as ready for archival */ + size = add_size(size, hash_estimate_size(16, sizeof(SegmentBoundaryEntry))); + /* * Note: we don't count ControlFileData, it comes out of the "slop factor" * added by CreateSharedMemoryAndSemaphores. This lets us use this @@ -5134,6 +5382,7 @@ XLOGShmemInit(void) char *allocptr; int i; ControlFileData *localControlFile; + HASHCTL info; #ifdef WAL_DEBUG @@ -5227,12 +5476,20 @@ XLOGShmemInit(void) XLogCtl->InstallXLogFileSegmentActive = false; XLogCtl->SharedPromoteIsTriggered = false; XLogCtl->WalWriterSleeping = false; + XLogCtl->lastNotifiedSeg = MaxXLogSegNo; SpinLockInit(&XLogCtl->Insert.insertpos_lck); SpinLockInit(&XLogCtl->info_lck); SpinLockInit(&XLogCtl->ulsn_lck); InitSharedLatch(&XLogCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogCtl->recoveryNotPausedCV); + + /* Initialize stuff for marking segments as ready for archival. */ + memset(&info, 0, sizeof(info)); + info.keysize = sizeof(XLogSegNo); + info.entrysize = sizeof(SegmentBoundaryEntry); + SegmentBoundaryMap = ShmemInitHash("Segment Boundary Table", 16, 16, &info, + HASH_ELEM | HASH_BLOBS); } /* @@ -7873,6 +8130,17 @@ StartupXLOG(void) XLogCtl->LogwrtRqst.Write = EndOfLog; XLogCtl->LogwrtRqst.Flush = EndOfLog; + /* + * Initialize XLogCtl->lastNotifiedSeg to the previous WAL file. + */ + if (XLogArchivingActive()) + { + XLogSegNo EndOfLogSeg; + + XLByteToSeg(EndOfLog, EndOfLogSeg, wal_segment_size); + SetLastNotifiedSegment(EndOfLogSeg - 1); + } + /* * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE * record before resource manager writes cleanup WAL records or checkpoint @@ -8000,7 +8268,7 @@ StartupXLOG(void) XLogArchiveCleanup(partialfname); durable_rename(origpath, partialpath, ERROR); - XLogArchiveNotify(partialfname); + XLogArchiveNotify(partialfname, true); } } } diff --git a/src/backend/access/transam/xlogarchive.c b/src/backend/access/transam/xlogarchive.c index 26b023e754..e9cac90b4a 100644 --- a/src/backend/access/transam/xlogarchive.c +++ b/src/backend/access/transam/xlogarchive.c @@ -433,7 +433,7 @@ KeepFileRestoredFromArchive(const char *path, const char *xlogfname) if (XLogArchiveMode != ARCHIVE_MODE_ALWAYS) XLogArchiveForceDone(xlogfname); else - XLogArchiveNotify(xlogfname); + XLogArchiveNotify(xlogfname, true); /* * If the existing file was replaced, since walsenders might have it open, @@ -464,7 +464,7 @@ KeepFileRestoredFromArchive(const char *path, const char *xlogfname) * then when complete, rename it to 0000000100000001000000C6.done */ void -XLogArchiveNotify(const char *xlog) +XLogArchiveNotify(const char *xlog, bool notify) { char archiveStatusPath[MAXPGPATH]; FILE *fd; @@ -489,8 +489,8 @@ XLogArchiveNotify(const char *xlog) return; } - /* Notify archiver that it's got something to do */ - if (IsUnderPostmaster) + /* If caller requested, notify archiver that it's got something to do */ + if (notify) PgArchWakeup(); } @@ -498,12 +498,12 @@ XLogArchiveNotify(const char *xlog) * Convenience routine to notify using segment number representation of filename */ void -XLogArchiveNotifySeg(XLogSegNo segno) +XLogArchiveNotifySeg(XLogSegNo segno, bool notify) { char xlog[MAXFNAMELEN]; XLogFileName(xlog, ThisTimeLineID, segno, wal_segment_size); - XLogArchiveNotify(xlog); + XLogArchiveNotify(xlog, notify); } /* @@ -608,7 +608,7 @@ XLogArchiveCheckDone(const char *xlog) return true; /* Retry creation of the .ready file */ - XLogArchiveNotify(xlog); + XLogArchiveNotify(xlog, true); return false; } diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 9a2bc37fd7..60de3be92c 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -622,7 +622,7 @@ WalReceiverMain(void) if (XLogArchiveMode != ARCHIVE_MODE_ALWAYS) XLogArchiveForceDone(xlogfname); else - XLogArchiveNotify(xlogfname); + XLogArchiveNotify(xlogfname, true); } recvFile = -1; @@ -760,7 +760,7 @@ WalRcvFetchTimeLineHistoryFiles(TimeLineID first, TimeLineID last) if (XLogArchiveMode != ARCHIVE_MODE_ALWAYS) XLogArchiveForceDone(fname); else - XLogArchiveNotify(fname); + XLogArchiveNotify(fname, true); pfree(fname); pfree(content); @@ -915,7 +915,7 @@ XLogWalRcvWrite(char *buf, Size nbytes, XLogRecPtr recptr) if (XLogArchiveMode != ARCHIVE_MODE_ALWAYS) XLogArchiveForceDone(xlogfname); else - XLogArchiveNotify(xlogfname); + XLogArchiveNotify(xlogfname, true); } recvFile = -1; diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt index 6c7cf6c295..14a742d655 100644 --- a/src/backend/storage/lmgr/lwlocknames.txt +++ b/src/backend/storage/lmgr/lwlocknames.txt @@ -53,3 +53,4 @@ XactTruncationLock 44 # 45 was XactTruncationLock until removal of BackendRandomLock WrapLimitsVacuumLock 46 NotifyQueueTailLock 47 +SegmentBoundaryLock 48 diff --git a/src/include/access/xlogarchive.h b/src/include/access/xlogarchive.h index 3edd1a976c..e25c3d8117 100644 --- a/src/include/access/xlogarchive.h +++ b/src/include/access/xlogarchive.h @@ -23,8 +23,8 @@ extern bool RestoreArchivedFile(char *path, const char *xlogfname, extern void ExecuteRecoveryCommand(const char *command, const char *commandName, bool failOnSignal); extern void KeepFileRestoredFromArchive(const char *path, const char *xlogfname); -extern void XLogArchiveNotify(const char *xlog); -extern void XLogArchiveNotifySeg(XLogSegNo segno); +extern void XLogArchiveNotify(const char *xlog, bool notify); +extern void XLogArchiveNotifySeg(XLogSegNo segno, bool notify); extern void XLogArchiveForceDone(const char *xlog); extern bool XLogArchiveCheckDone(const char *xlog); extern bool XLogArchiveIsBusy(const char *xlog); diff --git a/src/include/access/xlogdefs.h b/src/include/access/xlogdefs.h index 60348d1850..313c775e85 100644 --- a/src/include/access/xlogdefs.h +++ b/src/include/access/xlogdefs.h @@ -46,6 +46,7 @@ typedef uint64 XLogRecPtr; * XLogSegNo - physical log file sequence number. */ typedef uint64 XLogSegNo; +#define MaxXLogSegNo (0xFFFFFFFFFFFFFFFF) /* * TimeLineID (TLI) - identifies different database histories to prevent -- 2.16.6

view thread (127+ messages)  latest in thread

Message-ID: <2A53AFB0-9A44-4EFE-BA83-072D000D9319@amazon.com>
Permalink:  ../2A53AFB0-9A44-4EFE-BA83-072D000D9319@amazon.com/
Also on:    postgresql.org/message-id/2A53AFB0-9A44-4EFE-BA83-072D000D9319@amazon.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: bossartn@amazon.com, alvherre@alvh.no-ip.org, horikyota.ntt@gmail.com, x4mmm@yandex-team.ru, a.lubennikova@postgrespro.ru, hlinnaka@iki.fi, matsumura.ryo@fujitsu.com, masao.fujii@gmail.com
  Subject: Re: archive status ".ready" files may be created too early
  In-Reply-To: <2A53AFB0-9A44-4EFE-BA83-072D000D9319@amazon.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox