public inbox for [email protected]
help / color / mirror / Atom feedFrom: Bossart, Nathan <[email protected]>
To: Maxim Orlov <[email protected]>
To: [email protected] <[email protected]>
Subject: Re: Pre-allocating WAL files
Date: Fri, 8 Oct 2021 20:48:58 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <163352276350.15435.3988688227048304232.pgcf@coridan.postgresql.org>
References: <[email protected]>
<[email protected]>
<163352276350.15435.3988688227048304232.pgcf@coridan.postgresql.org>
On 10/6/21, 9:34 AM, "Bossart, Nathan" <[email protected]> wrote:
> I have been thinking about the right place to put this logic. My
> first thought is that it sounds like something that ought to go in the
> WAL writer process, but as Andres noted upthread, that is undesirable
> because it'll add latency for asynchronous commits. The checkpointer
> process seems to be another candidate, but I'm not totally sure how
> it'll fit in. My patch works by maintaining a small pool of pre-
> allocated segments that is quickly replenished whenever one is used.
> If the checkpointer is spending most of its time checkpointing, this
> small pool could remain empty for long periods of time. To keep pre-
> allocating WAL while we're checkpointing, perhaps we could add another
> function like CheckpointWriteDelay() that is called periodically.
> There still might be several operations in CheckPointGuts() that take
> a while and leave the segment pool empty, but maybe that's okay for
> now.
Here is a first attempt at adding the pre-allocation logic to the
checkpointer. I went ahead and just used CheckpointWriteDelay() for
pre-allocating during checkpoints. I've done a few pgbench runs, and
it seems to work pretty well. Initialization is around 15% faster,
and I'm seeing about a 5% increase in TPS with a simple-update
workload with wal_recycle turned off. Of course, these improvements
go away once segments can be recycled.
Nathan
Attachments:
[application/octet-stream] v6-0001-Move-WAL-segment-creation-logic-to-its-own-functi.patch (7.5K, ../[email protected]/2-v6-0001-Move-WAL-segment-creation-logic-to-its-own-functi.patch)
download | inline diff:
From 8974fbb7339ca8b75d2c500dde83a26324c92e51 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <[email protected]>
Date: Wed, 6 Oct 2021 17:16:34 +0000
Subject: [PATCH v6 1/2] Move WAL segment creation logic to its own function.
---
src/backend/access/transam/xlog.c | 103 +---------------------------------
src/backend/storage/file/fd.c | 114 ++++++++++++++++++++++++++++++++++++++
src/include/storage/fd.h | 1 +
3 files changed, 116 insertions(+), 102 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 26dcc00ac0..d0be54bc9b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -3284,11 +3284,9 @@ static int
XLogFileInitInternal(XLogSegNo logsegno, bool *added, char *path)
{
char tmppath[MAXPGPATH];
- PGAlignedXLogBlock zbuffer;
XLogSegNo installed_segno;
XLogSegNo max_segno;
int fd;
- int save_errno;
XLogFilePath(path, ThisTimeLineID, logsegno, wal_segment_size);
@@ -3316,106 +3314,7 @@ XLogFileInitInternal(XLogSegNo logsegno, bool *added, char *path)
elog(DEBUG2, "creating and filling new WAL file");
snprintf(tmppath, MAXPGPATH, XLOGDIR "/xlogtemp.%d", (int) getpid());
-
- unlink(tmppath);
-
- /* do not use get_sync_bit() here --- want to fsync only at end of fill */
- fd = BasicOpenFile(tmppath, O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
- if (fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m", tmppath)));
-
- memset(zbuffer.data, 0, XLOG_BLCKSZ);
-
- pgstat_report_wait_start(WAIT_EVENT_WAL_INIT_WRITE);
- save_errno = 0;
- if (wal_init_zero)
- {
- struct iovec iov[PG_IOV_MAX];
- int blocks;
-
- /*
- * Zero-fill the file. With this setting, we do this the hard way to
- * ensure that all the file space has really been allocated. On
- * platforms that allow "holes" in files, just seeking to the end
- * doesn't allocate intermediate space. This way, we know that we
- * have all the space and (after the fsync below) that all the
- * indirect blocks are down on disk. Therefore, fdatasync(2) or
- * O_DSYNC will be sufficient to sync future writes to the log file.
- */
-
- /* Prepare to write out a lot of copies of our zero buffer at once. */
- for (int i = 0; i < lengthof(iov); ++i)
- {
- iov[i].iov_base = zbuffer.data;
- iov[i].iov_len = XLOG_BLCKSZ;
- }
-
- /* Loop, writing as many blocks as we can for each system call. */
- blocks = wal_segment_size / XLOG_BLCKSZ;
- for (int i = 0; i < blocks;)
- {
- int iovcnt = Min(blocks - i, lengthof(iov));
- off_t offset = i * XLOG_BLCKSZ;
-
- if (pg_pwritev_with_retry(fd, iov, iovcnt, offset) < 0)
- {
- save_errno = errno;
- break;
- }
-
- i += iovcnt;
- }
- }
- else
- {
- /*
- * Otherwise, seeking to the end and writing a solitary byte is
- * enough.
- */
- errno = 0;
- if (pg_pwrite(fd, zbuffer.data, 1, wal_segment_size - 1) != 1)
- {
- /* if write didn't set errno, assume no disk space */
- save_errno = errno ? errno : ENOSPC;
- }
- }
- pgstat_report_wait_end();
-
- if (save_errno)
- {
- /*
- * If we fail to make the file, delete it to release disk space
- */
- unlink(tmppath);
-
- close(fd);
-
- errno = save_errno;
-
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmppath)));
- }
-
- pgstat_report_wait_start(WAIT_EVENT_WAL_INIT_SYNC);
- if (pg_fsync(fd) != 0)
- {
- int save_errno = errno;
-
- close(fd);
- errno = save_errno;
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not fsync file \"%s\": %m", tmppath)));
- }
- pgstat_report_wait_end();
-
- if (close(fd) != 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not close file \"%s\": %m", tmppath)));
+ CreateEmptyWalSegment(tmppath);
/*
* Now move the segment into place with its final name. Cope with
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index f9cda6906d..a44df86537 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -3870,3 +3870,117 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset)
return sum;
}
+
+/*
+ * CreateEmptyWalSegment
+ *
+ * Create a new file that can be used as a new WAL segment. The caller is
+ * responsible for installing the new file in pg_wal.
+ */
+void
+CreateEmptyWalSegment(const char *path)
+{
+ PGAlignedXLogBlock zbuffer;
+ int fd;
+ int save_errno;
+
+ unlink(path);
+
+ /* do not use get_sync_bit() here --- want to fsync only at end of fill */
+ fd = BasicOpenFile(path, O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
+ if (fd < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m", path)));
+
+ memset(zbuffer.data, 0, XLOG_BLCKSZ);
+
+ pgstat_report_wait_start(WAIT_EVENT_WAL_INIT_WRITE);
+ save_errno = 0;
+ if (wal_init_zero)
+ {
+ struct iovec iov[PG_IOV_MAX];
+ int blocks;
+
+ /*
+ * Zero-fill the file. With this setting, we do this the hard way to
+ * ensure that all the file space has really been allocated. On
+ * platforms that allow "holes" in files, just seeking to the end
+ * doesn't allocate intermediate space. This way, we know that we
+ * have all the space and (after the fsync below) that all the
+ * indirect blocks are down on disk. Therefore, fdatasync(2) or
+ * O_DSYNC will be sufficient to sync future writes to the log file.
+ */
+
+ /* Prepare to write out a lot of copies of our zero buffer at once. */
+ for (int i = 0; i < lengthof(iov); ++i)
+ {
+ iov[i].iov_base = zbuffer.data;
+ iov[i].iov_len = XLOG_BLCKSZ;
+ }
+
+ /* Loop, writing as many blocks as we can for each system call. */
+ blocks = wal_segment_size / XLOG_BLCKSZ;
+ for (int i = 0; i < blocks;)
+ {
+ int iovcnt = Min(blocks - i, lengthof(iov));
+ off_t offset = i * XLOG_BLCKSZ;
+
+ if (pg_pwritev_with_retry(fd, iov, iovcnt, offset) < 0)
+ {
+ save_errno = errno;
+ break;
+ }
+
+ i += iovcnt;
+ }
+ }
+ else
+ {
+ /*
+ * Otherwise, seeking to the end and writing a solitary byte is
+ * enough.
+ */
+ errno = 0;
+ if (pg_pwrite(fd, zbuffer.data, 1, wal_segment_size - 1) != 1)
+ {
+ /* if write didn't set errno, assume no disk space */
+ save_errno = errno ? errno : ENOSPC;
+ }
+ }
+ pgstat_report_wait_end();
+
+ if (save_errno)
+ {
+ /*
+ * If we fail to make the file, delete it to release disk space
+ */
+ unlink(path);
+
+ close(fd);
+
+ errno = save_errno;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", path)));
+ }
+
+ pgstat_report_wait_start(WAIT_EVENT_WAL_INIT_SYNC);
+ if (pg_fsync(fd) != 0)
+ {
+ int save_errno = errno;
+
+ close(fd);
+ errno = save_errno;
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not fsync file \"%s\": %m", path)));
+ }
+ pgstat_report_wait_end();
+
+ if (close(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m", path)));
+}
diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h
index 34602ae006..abcd74bd51 100644
--- a/src/include/storage/fd.h
+++ b/src/include/storage/fd.h
@@ -190,6 +190,7 @@ extern int durable_unlink(const char *fname, int loglevel);
extern int durable_rename_excl(const char *oldfile, const char *newfile, int loglevel);
extern void SyncDataDirectory(void);
extern int data_sync_elevel(int elevel);
+extern void CreateEmptyWalSegment(const char *path);
/* Filename components */
#define PG_TEMP_FILES_DIR "pgsql_tmp"
--
2.16.6
[application/octet-stream] v6-0002-WAL-segment-pre-allocation.patch (24.0K, ../[email protected]/3-v6-0002-WAL-segment-pre-allocation.patch)
download | inline diff:
From 78329f1a6b51639eb8f69426ef127091b44c3895 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <[email protected]>
Date: Fri, 8 Oct 2021 20:22:23 +0000
Subject: [PATCH v6 2/2] WAL segment pre-allocation.
---
doc/src/sgml/config.sgml | 23 +++
src/backend/access/transam/xlog.c | 60 ++++++-
src/backend/postmaster/checkpointer.c | 234 ++++++++++++++++++++++++++
src/backend/replication/basebackup.c | 6 +-
src/backend/storage/file/fd.c | 22 ++-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/utils/misc/guc.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/initdb/initdb.c | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 19 ++-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/bin/pg_resetwal/pg_resetwal.c | 48 ++++++
src/include/postmaster/bgwriter.h | 5 +
src/include/storage/fd.h | 2 +-
14 files changed, 417 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 0a8e35c59f..0e66e415ff 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3203,6 +3203,29 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-wal-preallocate-max-size" xreflabel="wal_preallocate_max_size">
+ <term><varname>wal_preallocate_max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>wal_preallocate_max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ The maximum amount of WAL to pre-allocate for use when a new WAL segment
+ must be initialized. Setting this parameter to a size less than the WAL
+ segment size disables this behavior. The default value is 64 megabytes
+ (<literal>64MB</literal>). This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command line.
+ </para>
+
+ <para>
+ WAL pre-allocation may improve performance in scenarios where new WAL
+ segments must be created (e.g., during checkpoints when WAL segments
+ cannot be recycled).
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-wal-buffers" xreflabel="wal_buffers">
<term><varname>wal_buffers</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index d0be54bc9b..e0b84321fb 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -3287,6 +3287,8 @@ XLogFileInitInternal(XLogSegNo logsegno, bool *added, char *path)
XLogSegNo installed_segno;
XLogSegNo max_segno;
int fd;
+ int prealloc_segs;
+ bool found_prealloc = false;
XLogFilePath(path, ThisTimeLineID, logsegno, wal_segment_size);
@@ -3306,15 +3308,45 @@ XLogFileInitInternal(XLogSegNo logsegno, bool *added, char *path)
return fd;
/*
- * Initialize an empty (all zeroes) segment. NOTE: it is possible that
- * another process is doing the same thing. If so, we will end up
- * pre-creating an extra log segment. That seems OK, and better than
- * holding the lock throughout this lengthy process.
+ * Try to use a pre-allocated segment, if one exists. If none are
+ * available, we fall back to creating a new segment on our own.
+ *
+ * Note that we still look for a pre-allocated segment even if the pre-
+ * allocation functionality is disabled via the GUCs. This ensures that any
+ * pre-allocated segments left over after turning off the pre-allocation
+ * functionality are still eligible for use.
*/
- elog(DEBUG2, "creating and filling new WAL file");
+ LWLockAcquire(WALPreallocationLock, LW_EXCLUSIVE);
+ prealloc_segs = GetNumPreallocatedWalSegs();
+ if (prealloc_segs > 0)
+ {
+ elog(DEBUG2, "using pre-allocated WAL file");
- snprintf(tmppath, MAXPGPATH, XLOGDIR "/xlogtemp.%d", (int) getpid());
- CreateEmptyWalSegment(tmppath);
+ found_prealloc = true;
+ prealloc_segs--;
+ SetNumPreallocatedWalSegs(prealloc_segs);
+ snprintf(tmppath, MAXPGPATH, "%s/preallocated_segments/xlogtemp.%d",
+ XLOGDIR, prealloc_segs);
+ }
+ else
+ {
+ /*
+ * We're not using a pre-allocated segment, so there's no need to keep
+ * holding the WALPreallocationLock.
+ */
+ LWLockRelease(WALPreallocationLock);
+
+ /*
+ * Initialize an empty (all zeroes) segment. NOTE: it is possible that
+ * another process is doing the same thing. If so, we will end up
+ * pre-creating an extra log segment. That seems OK, and better than
+ * holding the lock throughout this lengthy process.
+ */
+ elog(DEBUG2, "creating and filling new WAL file");
+
+ snprintf(tmppath, MAXPGPATH, XLOGDIR "/xlogtemp.%d", (int) getpid());
+ CreateEmptyWalSegment(tmppath, ERROR);
+ }
/*
* Now move the segment into place with its final name. Cope with
@@ -3336,7 +3368,7 @@ XLogFileInitInternal(XLogSegNo logsegno, bool *added, char *path)
if (InstallXLogFileSegment(&installed_segno, tmppath, true, max_segno))
{
*added = true;
- elog(DEBUG2, "done creating and filling new WAL file");
+ elog(DEBUG2, "done installing new WAL file");
}
else
{
@@ -3349,6 +3381,18 @@ XLogFileInitInternal(XLogSegNo logsegno, bool *added, char *path)
elog(DEBUG2, "abandoned new WAL file");
}
+ /*
+ * If we are using a pre-allocated segment, we've been holding onto the
+ * WALPreallocationLock all this time so that the checkpointer process can't
+ * overwrite the file before we've installed it.
+ *
+ * While we're at it, also nudge the checkpointer process so that it pre-
+ * allocates new segments if possible.
+ */
+ if (found_prealloc)
+ LWLockRelease(WALPreallocationLock);
+ RequestWalPreallocation();
+
return -1;
}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..3237595dbf 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -127,6 +127,9 @@ typedef struct
uint32 num_backend_writes; /* counts user backend buffer writes */
uint32 num_backend_fsync; /* counts user backend fsync calls */
+ bool wal_prealloc_requested; /* protected by ckpt_lck */
+ int num_prealloc_segs; /* protected by WALPreallocationLock */
+
int num_requests; /* current # of requests */
int max_requests; /* allocated array size */
CheckpointerRequest requests[FLEXIBLE_ARRAY_MEMBER];
@@ -143,6 +146,7 @@ static CheckpointerShmemStruct *CheckpointerShmem;
int CheckPointTimeout = 300;
int CheckPointWarning = 30;
double CheckPointCompletionTarget = 0.9;
+int wal_prealloc_max_size_mb = 0;
/*
* Private state
@@ -165,6 +169,10 @@ static bool IsCheckpointOnSchedule(double progress);
static bool ImmediateCheckpointRequested(void);
static bool CompactCheckpointerRequestQueue(void);
static void UpdateSharedMemoryConfig(void);
+static void ScanForExistingPreallocatedSegments(void);
+static bool InstallPreallocatedWalSeg(const char *path);
+static void DoWalPreAllocation(void);
+static int GetMaxPreallocatedWalSegs(void);
/* Signal handlers */
static void ReqCheckpointHandler(SIGNAL_ARGS);
@@ -329,6 +337,11 @@ CheckpointerMain(void)
*/
ProcGlobal->checkpointerLatch = &MyProc->procLatch;
+ /*
+ * Look for any leftover pre-allocated segments we can use.
+ */
+ ScanForExistingPreallocatedSegments();
+
/*
* Loop forever
*/
@@ -349,6 +362,11 @@ CheckpointerMain(void)
AbsorbSyncRequests();
HandleCheckpointerInterrupts();
+ /*
+ * First do WAL pre-allocation.
+ */
+ DoWalPreAllocation();
+
/*
* Detect a pending checkpoint request by checking whether the flags
* word in shared memory is nonzero. We shouldn't need to acquire the
@@ -507,6 +525,12 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
continue;
+ /*
+ * Also skip sleeping if WAL pre-allocation has been requested.
+ */
+ if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->wal_prealloc_requested)
+ continue;
+
/*
* Sleep until we are signaled or it's time for another checkpoint or
* xlog file switch.
@@ -682,6 +706,8 @@ ImmediateCheckpointRequested(void)
*
* 'progress' is an estimate of how much of the work has been done, as a
* fraction between 0.0 meaning none, and 1.0 meaning all done.
+ *
+ * This function also takes care of handling any WAL pre-allocation requests.
*/
void
CheckpointWriteDelay(int flags, double progress)
@@ -692,6 +718,15 @@ CheckpointWriteDelay(int flags, double progress)
if (!AmCheckpointerProcess())
return;
+ /*
+ * WAL pre-allocation has nothing to do with throttling BufferSync()'s write
+ * rate. However, since this function is called frequently during
+ * checkpoints, it is a convenient place to handle any pending WAL pre-
+ * allocation requests.
+ */
+ if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->wal_prealloc_requested)
+ DoWalPreAllocation();
+
/*
* Perform the usual duties and take a nap, unless we're behind schedule,
* in which case we just try to catch up as quickly as possible.
@@ -823,6 +858,203 @@ IsCheckpointOnSchedule(double progress)
return true;
}
+/*
+ * GetNumPreallocatedWalSegs
+ *
+ * Returns the number of pre-allocated WAL segments in the preallocated_segments
+ * directory. Callers are expected to hold the WALPreallocationLock.
+ */
+int
+GetNumPreallocatedWalSegs(void)
+{
+ Assert(LWLockHeldByMe(WALPreallocationLock));
+ return CheckpointerShmem->num_prealloc_segs;
+}
+
+/*
+ * SetNumPreallocatedWalSegs
+ *
+ * Sets the number of pre-allocated WAL segments in the preallocated_segments
+ * directory. Callers are expected to hold the WALPreallocationLock
+ * exclusively.
+ */
+void
+SetNumPreallocatedWalSegs(int i)
+{
+ Assert(LWLockHeldByMeInMode(WALPreallocationLock, LW_EXCLUSIVE));
+ CheckpointerShmem->num_prealloc_segs = i;
+}
+
+/*
+ * ScanForExistingPreallocatedSegments
+ *
+ * This function searches through pg_wal/preallocated_segments for any segments
+ * that were left over and sets the tracking variable in shared memory
+ * accordingly.
+ */
+static void
+ScanForExistingPreallocatedSegments(void)
+{
+ int i = 0;
+
+ /*
+ * fsync the preallocated_segments directory in case any renames have yet to
+ * be flushed to disk.
+ */
+ fsync_fname_ext(XLOGDIR "/preallocated_segments", true, false, FATAL);
+
+ /*
+ * Gather all the preallocated segments we can find.
+ */
+ while (true)
+ {
+ FILE *fd;
+ char path[MAXPGPATH];
+
+ snprintf(path, MAXPGPATH, "%s/preallocated_segments/xlogtemp.%d",
+ XLOGDIR, i);
+
+ fd = AllocateFile(path, "r");
+ if (fd != NULL)
+ {
+ FreeFile(fd);
+ i++;
+ }
+ else
+ {
+ if (errno != ENOENT)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));
+ break;
+ }
+ }
+
+ LWLockAcquire(WALPreallocationLock, LW_EXCLUSIVE);
+ SetNumPreallocatedWalSegs(i);
+ LWLockRelease(WALPreallocationLock);
+
+ elog(DEBUG2, "found %d preallocated segments during startup", i);
+}
+
+/*
+ * InstallPreallocatedWalSeg
+ *
+ * Renames the file at "path" to the next open pre-allocated segment slot and
+ * bumps up num_prealloc_segs. If there is a problem, a WARNING is emitted, we
+ * attempt to delete the file, and false is returned. Otherwise, true is
+ * returned.
+ */
+static bool
+InstallPreallocatedWalSeg(const char *path)
+{
+ char newpath[MAXPGPATH];
+ int rc;
+
+ LWLockAcquire(WALPreallocationLock, LW_EXCLUSIVE);
+
+ snprintf(newpath, MAXPGPATH, "%s/preallocated_segments/xlogtemp.%d",
+ XLOGDIR, CheckpointerShmem->num_prealloc_segs);
+
+ rc = durable_rename(path, newpath, DEBUG1);
+ if (rc == 0)
+ CheckpointerShmem->num_prealloc_segs++;
+ else
+ {
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("file \"%s\" could not be renamed to \"%s\": %m",
+ path, newpath)));
+
+ (void) durable_unlink(path, DEBUG1);
+ (void) durable_unlink(newpath, DEBUG1);
+ }
+
+ LWLockRelease(WALPreallocationLock);
+
+ return (rc == 0);
+}
+
+/*
+ * DoWalPreAllocation
+ *
+ * Tries to allocate up to wal_preallocate_max_size worth of WAL.
+ */
+static void
+DoWalPreAllocation(void)
+{
+ int segs_to_prealloc;
+
+ /*
+ * Reset the request flag.
+ */
+ SpinLockAcquire(&CheckpointerShmem->ckpt_lck);
+ CheckpointerShmem->wal_prealloc_requested = false;
+ SpinLockRelease(&CheckpointerShmem->ckpt_lck);
+
+ /*
+ * Determine how many segments to pre-allocate.
+ */
+ LWLockAcquire(WALPreallocationLock, LW_SHARED);
+ segs_to_prealloc = GetMaxPreallocatedWalSegs() - GetNumPreallocatedWalSegs();
+ LWLockRelease(WALPreallocationLock);
+
+ /*
+ * Do the pre-allocation.
+ */
+ for (int i = 0; i < segs_to_prealloc; i++)
+ {
+ char tmppath[MAXPGPATH];
+
+ snprintf(tmppath, MAXPGPATH, "%s/preallocated_segments/xlogtemp", XLOGDIR);
+
+ if (!CreateEmptyWalSegment(tmppath, WARNING) ||
+ !InstallPreallocatedWalSeg(tmppath))
+ {
+ elog(DEBUG2, "failed to pre-allocate WAL segment");
+ return;
+ }
+ else
+ elog(DEBUG2, "pre-allocated WAL segment");
+
+ /* Check for barrier events. */
+ if (ProcSignalBarrierPending)
+ ProcessProcSignalBarrier();
+
+ if (ShutdownRequestPending)
+ return;
+ }
+}
+
+/*
+ * GetMaxPreallocatedWalSegs
+ *
+ * Returns the maximum number of pre-allocated WAL segments to create based on
+ * the current value of wal_preallocate_max_size.
+ */
+static int
+GetMaxPreallocatedWalSegs(void)
+{
+ return wal_prealloc_max_size_mb / (wal_segment_size / (1024 * 1024));
+}
+
+/*
+ * RequestWalPreallocation
+ *
+ * Requests that more segments be pre-allocated for future use.
+ */
+void
+RequestWalPreallocation(void)
+{
+ SpinLockAcquire(&CheckpointerShmem->ckpt_lck);
+ CheckpointerShmem->wal_prealloc_requested = true;
+ SpinLockRelease(&CheckpointerShmem->ckpt_lck);
+
+ if (ProcGlobal->checkpointerLatch &&
+ GetMaxPreallocatedWalSegs() > 0)
+ SetLatch(ProcGlobal->checkpointerLatch);
+}
+
/* --------------------------------
* signal handler routines
@@ -896,6 +1128,8 @@ CheckpointerShmemInit(void)
CheckpointerShmem->max_requests = NBuffers;
ConditionVariableInit(&CheckpointerShmem->start_cv);
ConditionVariableInit(&CheckpointerShmem->done_cv);
+ CheckpointerShmem->wal_prealloc_requested = false;
+ CheckpointerShmem->num_prealloc_segs = 0;
}
}
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 4c97ab7b5a..e84288a8a7 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1412,11 +1412,13 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
size += _tarWriteDir(pathbuf, basepathlen, &statbuf, sizeonly);
/*
- * Also send archive_status directory (by hackishly reusing
- * statbuf from above ...).
+ * Also send archive_status and preallocated_segments (by hackishly
+ * reusing statbuf from above ...).
*/
size += _tarWriteHeader("./pg_wal/archive_status", NULL, &statbuf,
sizeonly);
+ size += _tarWriteHeader("./pg_wal/preallocated_segments", NULL, &statbuf,
+ sizeonly);
continue; /* don't recurse into pg_wal */
}
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index a44df86537..e5425597df 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -3877,8 +3877,8 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset)
* Create a new file that can be used as a new WAL segment. The caller is
* responsible for installing the new file in pg_wal.
*/
-void
-CreateEmptyWalSegment(const char *path)
+bool
+CreateEmptyWalSegment(const char *path, int elevel)
{
PGAlignedXLogBlock zbuffer;
int fd;
@@ -3889,9 +3889,12 @@ CreateEmptyWalSegment(const char *path)
/* do not use get_sync_bit() here --- want to fsync only at end of fill */
fd = BasicOpenFile(path, O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
if (fd < 0)
- ereport(ERROR,
+ {
+ ereport(elevel,
(errcode_for_file_access(),
errmsg("could not create file \"%s\": %m", path)));
+ return false;
+ }
memset(zbuffer.data, 0, XLOG_BLCKSZ);
@@ -3961,9 +3964,10 @@ CreateEmptyWalSegment(const char *path)
errno = save_errno;
- ereport(ERROR,
+ ereport(elevel,
(errcode_for_file_access(),
errmsg("could not write to file \"%s\": %m", path)));
+ return false;
}
pgstat_report_wait_start(WAIT_EVENT_WAL_INIT_SYNC);
@@ -3973,14 +3977,20 @@ CreateEmptyWalSegment(const char *path)
close(fd);
errno = save_errno;
- ereport(ERROR,
+ ereport(elevel,
(errcode_for_file_access(),
errmsg("could not fsync file \"%s\": %m", path)));
+ return false;
}
pgstat_report_wait_end();
if (close(fd) != 0)
- ereport(ERROR,
+ {
+ ereport(elevel,
(errcode_for_file_access(),
errmsg("could not close file \"%s\": %m", path)));
+ return false;
+ }
+
+ return true;
}
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 6c7cf6c295..212bda8e7d 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock 44
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+WALPreallocationLock 48
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index d2ce4a8450..4d3274a70c 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2840,6 +2840,17 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"wal_preallocate_max_size", PGC_SIGHUP, WAL_SETTINGS,
+ gettext_noop("Sets the maximum amount of WAL to pre-allocate."),
+ NULL,
+ GUC_UNIT_MB
+ },
+ &wal_prealloc_max_size_mb,
+ 64, 0, 102400,
+ NULL, NULL, NULL
+ },
+
{
{"wal_buffers", PGC_POSTMASTER, WAL_SETTINGS,
gettext_noop("Sets the number of disk-page buffers in shared memory for WAL."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 3fe9a53cb3..309745e932 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -223,6 +223,7 @@
# off, pglz, lz4, or on
#wal_init_zero = on # zero-fill new WAL files
#wal_recycle = on # recycle WAL files
+#wal_preallocate_max_size = 64MB # amount of WAL to pre-allocate
#wal_buffers = -1 # min 32kB, -1 sets based on shared_buffers
# (change requires restart)
#wal_writer_delay = 200ms # 1-10000 milliseconds
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 1ed4808d53..aa52aaeffb 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -208,6 +208,7 @@ static char *extra_options = "";
static const char *const subdirs[] = {
"global",
"pg_wal/archive_status",
+ "pg_wal/preallocated_segments",
"pg_commit_ts",
"pg_dynshmem",
"pg_notify",
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 27ee6394cf..0623cfe4b3 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -680,6 +680,22 @@ StartLogStreamer(char *startpos, uint32 timeline, char *sysidentifier)
pg_log_error("could not create directory \"%s\": %m", statusdir);
exit(1);
}
+
+ /*
+ * Also create pg_wal/preallocated_segments if necessary.
+ */
+ if (PQserverVersion(conn) >= 150000)
+ {
+ char prealloc_dir[MAXPGPATH];
+
+ snprintf(prealloc_dir, sizeof(prealloc_dir), "%s/pg_wal/preallocated_segments",
+ basedir);
+ if (pg_mkdir_p(prealloc_dir, pg_dir_create_mode) != 0 && errno != EEXIST)
+ {
+ pg_log_error("could not create directory \"%s\": %m", prealloc_dir);
+ exit(1);
+ }
+ }
}
/*
@@ -1611,7 +1627,8 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
*/
if (!((pg_str_endswith(state->filename, "/pg_wal") ||
pg_str_endswith(state->filename, "/pg_xlog") ||
- pg_str_endswith(state->filename, "/archive_status")) &&
+ pg_str_endswith(state->filename, "/archive_status") ||
+ pg_str_endswith(state->filename, "/preallocated_segments")) &&
errno == EEXIST))
{
pg_log_error("could not create directory \"%s\": %m",
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index a2cb2a7679..43e1e3e89d 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -120,10 +120,10 @@ SKIP:
"check backup dir permissions");
}
-# Only archive_status directory should be copied in pg_wal/.
+# Only archive_status and preallocated_segments directories should be copied in pg_wal/.
is_deeply(
[ sort(slurp_dir("$tempdir/backup/pg_wal/")) ],
- [ sort qw(. .. archive_status) ],
+ [ sort qw(. .. archive_status preallocated_segments) ],
'no WAL files copied');
# Contents of these directories should not be copied.
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index 84b6f70ad6..ca09ad72d2 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -85,6 +85,7 @@ static void RewriteControlFile(void);
static void FindEndOfXLOG(void);
static void KillExistingXLOG(void);
static void KillExistingArchiveStatus(void);
+static void KillExistingPreallocatedSegments(void);
static void WriteEmptyXLOG(void);
static void usage(void);
@@ -530,6 +531,7 @@ main(int argc, char *argv[])
RewriteControlFile();
KillExistingXLOG();
KillExistingArchiveStatus();
+ KillExistingPreallocatedSegments();
WriteEmptyXLOG();
printf(_("Write-ahead log reset\n"));
@@ -1120,6 +1122,52 @@ KillExistingArchiveStatus(void)
}
+/*
+ * Remove existing preallocated segments
+ */
+static void
+KillExistingPreallocatedSegments(void)
+{
+#define PREALLOCSEGDIR XLOGDIR "/preallocated_segments"
+
+ DIR *xldir;
+ struct dirent *xlde;
+ char path[MAXPGPATH + sizeof(PREALLOCSEGDIR)];
+
+ xldir = opendir(PREALLOCSEGDIR);
+ if (xldir == NULL)
+ {
+ pg_log_error("could not open directory \"%s\": %m", PREALLOCSEGDIR);
+ exit(1);
+ }
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ {
+ if (strncmp(xlde->d_name, "xlogtemp", strlen("xlogtemp")) == 0)
+ {
+ snprintf(path, sizeof(path), "%s/%s", PREALLOCSEGDIR, xlde->d_name);
+ if (unlink(path) < 0)
+ {
+ pg_log_error("could not delete file \"%s\": %m", path);
+ exit(1);
+ }
+ }
+ }
+
+ if (errno)
+ {
+ pg_log_error("could not read directory \"%s\": %m", PREALLOCSEGDIR);
+ exit(1);
+ }
+
+ if (closedir(xldir))
+ {
+ pg_log_error("could not close directory \"%s\": %m", PREALLOCSEGDIR);
+ exit(1);
+ }
+}
+
+
/*
* Write an empty XLOG file, containing only the checkpoint record
* already set up in ControlFile.
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index c430b1b236..8303eb48cb 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -26,6 +26,7 @@ extern int BgWriterDelay;
extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
+extern int wal_prealloc_max_size_mb;
extern void BackgroundWriterMain(void) pg_attribute_noreturn();
extern void CheckpointerMain(void) pg_attribute_noreturn();
@@ -42,4 +43,8 @@ extern void CheckpointerShmemInit(void);
extern bool FirstCallSinceLastCheckpoint(void);
+extern int GetNumPreallocatedWalSegs(void);
+extern void SetNumPreallocatedWalSegs(int i);
+extern void RequestWalPreallocation(void);
+
#endif /* _BGWRITER_H */
diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h
index abcd74bd51..2d174509c5 100644
--- a/src/include/storage/fd.h
+++ b/src/include/storage/fd.h
@@ -190,7 +190,7 @@ extern int durable_unlink(const char *fname, int loglevel);
extern int durable_rename_excl(const char *oldfile, const char *newfile, int loglevel);
extern void SyncDataDirectory(void);
extern int data_sync_elevel(int elevel);
-extern void CreateEmptyWalSegment(const char *path);
+extern bool CreateEmptyWalSegment(const char *path, int elevel);
/* Filename components */
#define PG_TEMP_FILES_DIR "pgsql_tmp"
--
2.16.6
view thread (19+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Pre-allocating WAL files
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox